Philip Shapira, School of Public Policy, Georgia Institute of Technology, Atlanta, Georgia 30332-0345 USA, email: philip.shapira@pubpolicy.gatech.edu ; Jan Youtie, Economic Development Institute, Georgia Institute of Technology, Atlanta, Georgia 30332-0640 USA, email: jan.youtie@edi.gatech.edu
April 1995
ABSTRACT | INTRODUCTION | THE MODEL | KEY ISSUES | RESULTS | CONCLUSION | ACKNOWLEDGEMENT | REFERENCES
ABSTRACT
This paper presents an attempt to more fully account for and estimate the
costs and benefits of one manufacturing extension program, the Georgia
Manufacturing Extension Alliance (GMEA). Drawing on data from GMEA's first
year of activities, the results indicate that GMEA leverages relatively
high levels of private investment which, in turn, is likely to lead to
favorable and positive public and private returns over time.
Introduction
Over the past few years, there has been a considerable growth policies
and programs to improve the industrial performance of U.S. firms, especially
small and mid-sized companies (those with 500 or fewer employees). (Rosenfeld
1992; Simons 1993; Shapira et al. 1995) Increased state, local and private
resources for industrial modernization have been supplemented by new federal
efforts, organized through the Department of Commerce's Manufacturing Extension
Partnership. (National Institute of Standards and Technology 1996). Such
policies and programs aim to assist firms through the provision of technical
assistance, information, assessments and problem solving, business strategy
development, training, and networking. These efforts generally focus on
the deployment of existing and known technologies, including proven management
and quality systems. As manufacturing extension and other technology transfer
programs have increased in scale and resources, there has been increased
interest in trying to assess not only outcomes from individual firms, but
also economic and regional impacts and returns on the public investment.
In early studies, the inadequate treatment of the full range of costs and
benefits and the lax use of standard multiplier ratios have led to implausibly
high return ratios. For example, at times the monetary values of extra
sales and new capital expenditures attributed to a modernization program
are added together as part of analyses of program benefits, even though
the latter is a cost to the investing firm (although presumably, the firm
anticipates return on the capital expenditures). In other instances, only
a portion of program costs are included in the analysis. For example, states
and localities may treat federal funds as money without costs. For example,
the NIST Manufacturing Extension Partnership published an estimate that
there is a private return of over $7 for each federal dollar invested in
Manufacturing Technology Centers (MTCs) between 1989 and 1992--a calculation
which both excludes state and local costs and mingles private costs and
benefits (Shapira et al. 1996).
The Model
This report aims to operationalize a model which more fully and completely
accounts for the costs and benefits derived from industrial modernization
and technology transfer programs. (Feller 1995; Feller and Anderson ).
The benefit-cost model used in this study is structured as follows:
B:C = (Private Returns + Public Returns) (Private Investment + Public Investment)
where
B:C = ratio
of benefits to costs
To operationalize the model, the analysis draws on the experience of the Georgia Manufacturing Extension Alliance (GMEA) of the NIST Manufacturing Extension Partnership, for whom the authors direct an evaluation element.(Shapira and Youtie 1995). Table 1 shows how the elements of the model have been operationalized. Much of the information is drawn from a customer satisfaction survey administered sometime after project closure--generally one to six months afterwards--to all projects receiving eight or more hours of service. The survey is sent by mail to the company contact, and is followed up by mail and telephone if necessary. In addition, to satisfaction and a question asking whether or not the company intends to take action as a result of the services, there are questions asking whether the customer anticipates impacts, and estimates measuring the projected dollar value of the assistance and services, along several dimensions:
Category Factor Source
Private investment Customer staff time Customer evaluation of
commitment services survey
Increased capital spending GMEA financial records
Fees
Private returns Increase in sales Customer evaluation of
Savings in labor, services survey
materials, energy, or
other costs
Reductions in amount of
inventory carried
Avoidance of capital
spending
Public investment Federal, state and local GMEA financial records
program expenditures,
excluding in-kind match
Public returns Federal, state, and local Customer evaluation of
taxes paid by companies services survey
and their employees, State Input-Output Model
estimated from sales
increases or job
creation/retention
There has been a debate in the evaluation community about how to treat many of the elements of the cost benefit analysis. Table 2 summarizes these issues and their treatment in this analysis.
Table 2. Treatment of Cost Benefit Elements
Issue Treatment
Anticipated . Use of median and adjusted mean to develop range
returns/investments . One-year follow-up survey suggests little
difference overall
Sales increases . Sales adjusted by ratio of value-added to shipments
from 1992 Census of Manufactures (.451).
. Sensitivity analysis
Treatment of returns . One-time impacts (capital spending)
and investments over . Multi-year impacts (Sales, operating cost savings,
time inventory):
--3-year time frame
--declining impacts in latter years
"Zero-sum outcome" Adjustment based on survey - Georgia manufacturing
(geographical shift of sales volume by market
benefits)
Multipliers . Only first round of benefits
Qualitative benefits . Quantitative benefit focus
Anticipated impacts. One issue related to the use of "anticipated" impacts. Information about private returns and investment draws from the customer evaluation of services survey, which asked respondents to report "anticipated" impacts resulting from GMEA assistance and services. It is thought that when close to the point of service delivery (in this case, it is 30 to 180 days), manufacturers may over-estimate anticipated effects of public services. The analysis addresses the possible over-estimate by developing a "low" range of benefits and costs, as well as a "high" range, using the median and a "trimmed" mean which excludes outliers three standard deviations from the mean. The project team also hoped that the use of current and anticipated benefits and investments from companies served by GMEA would compensate for this problem in that company overestimates of benefits might be offset by corresponding overestimates of capital investment factored into the cost side of the analysis. A subsequent round of data collection has since been completed in which the project team conducted a one-year follow-up survey to obtain a better measure of actual rather than anticipated benefits and investments. The subsequent analysis found that for the average project, actual sales and cost savings tended to be less than the companies had anticipated, although jobs added or saved were the same or higher than anticipated. It was also found that costs--that is capital investments and company time investments--were higher than anticipated. The distribution of projects by impact was not normal--there were a few outlying, high impact projects which did not follow the trend seen in the average project. Sales and jobs were both higher than anticipated for the outlying, high impact cases.
Treatment of sales increases. Sales increases should be adjusted for the fact that firms retain only a portion of the increase after material purchases, and other costs. The model adjusts for value-added by imputing a value based on the 1992 Census of Manufacturers. A related issue concerns the full estimation of the net impact of the program on consumer surplus. Because GMEA works with a multitude of firms operating a further multitude of markets, consumer surplus would be very hard to measure. This analysis assumes that consumer surplus would be some fraction of value added. The project team conducted a sensitivity analysis which adjusts value-added downwards at different levels as a proxy for consumer surplus.
Treatment of model elements over time. In the benefit-cost analysis, we thus identify two types of impacts: one time (or one-year) and multi-year. Examples of one-time (or one-year) impacts are program costs, the value of private staff time, and project fee revenues. Capital spending and capital outlays avoided are both treated as one-time impacts. Multi-year impacts include savings in labor, materials, energy and other costs, inventory savings (interest savings on borrowing made unnecessary), and increases in sales (imputed value-added). For the multi-year impacts, we calculate the present value. The analysis assumes that such flows will extend over three years, but with a declining distribution which reduces each subsequent year's benefits by half. The justification for this is that the impacts which can be credited to the program need to have a fixed time horizon (they cannot be assumed to occur indefinitely), and we assume the further away in time from the point of service, the lower the likelihood of a continued impact (and the greater the chance of other factors affecting changes in a company). The present value of these flows is calculated using a prime plus two percent discount rate for private returns and investments, because most small and medium-sized establishments do not borrow at the prime rate.
Zero-sum outcomes. There has been debate in the industrial extension and evaluation community about how to treat company estimates of benefits as a result of industrial extension services. In part, this depends on geographical perspective. While sales increases may be associated with new exports, replaced imports, or changes in customer demand, it is also possible that a portion of the sales increase attributed by a company to industrial extension assistance is "taken" from other competing U.S. companies. To the extent that there is a shift of sales from one company to another, this may nullify the effect of industrial extension from a national perspective. This is popularly called a "zero-sum" outcome. (Wood 1994; Chrisman and McMullan 1996.) Based on statewide survey data of the percent manufacturing sales volume within state, elsewhere in the U.S., and export sales, the project team elected to use an adjustment factor of about 30 percent. This represents a national perspective. If a Georgia state perspective were adopted, the proportion of elsewhere in the U.S. sales counted as new might be raised to a higher level.
Multipliers. Where economic impact studies have been undertaken, efforts have sometimes been made to estimate the multipliers or additional spending or income generated within a regional economy as a result of direct impacts from assistance provided to manufacturers and the implied total return on public investment. Inevitably attempts to determine the total impacts of industrial modernization programs raise a series of methodological and measurement issues common to all economic impact analyses of technology and economic development programs. These include problems of attribution of cause and effect against the larger scale and other influences of the region's economy. This analysis only includes the first round of direct benefits, analysis, although a further round of surveying may allow us account for benefits leakage by looking at use of outsourcing.
Qualitative benefits. This analysis focuses on quantifiable benefits. It avoids the difficult process of estimating "soft," albeit important, benefits such as improved information and better decision-making capabilities.
Results
The present value per project cost is estimated at between $27,700
to $102,100, of which public project costs are estimated at $7,100. This
implies a significant GMEA public leveraging effect on private resources.
Manufacturing customers invest from $3 to $13 in their improvement efforts
for every public dollar spent. For a given project, public benefits run
from $10,600 to $13,600. Private returns per project range from $65,200
to $106,600 (recall, these include the present values of multi-year benefits).
For every public dollar flowing to government, private firms gain $6 to
$8 in benefits. The analysis calculates an implied private payback period
(assuming they fall evenly by month, which they probably would not). The
results indicate that the typical payback period for manufacturers using
GMEA assistance and services ranges from six to 22 months. The present
value of total private investment ranges from $11 million to $50 million.
Public costs are under $4 million. The present value of total private returns
ranges from $35 million to $57 million. Public returns fall in the $5 to
$7 million range. Net private returns (returns less investment) range from
$6 million to $24 million (present value). Net public returns are $2 million
to $3 million. The total (public and private) net returns of the program
range from $10 million to $26 million (present value), implying a benefit-cost
ratio from 1.2 to 2.7. (See Table
3.)
Table 3. GMEA Cost Benefit Analysis
Summary Estimates and Ratios (Year 1 Projects)
(Costs and benefits in thousands of dollars)
Total Per Project
Method A Method B Method A Method B
Costs Private Investment 10,923.8 50,520.5 20.5 95.0
Public Investment 3,797.7 3,797.7 7.1 7.1
Total Investment 14,721.5 54,318.3 27.7 102.1
Benefits Private Returns 34,707.5 56,707.5 65.2 106.6
Public Returns 5,629.4 7,259.5 10.6 13.6
Total Returns 40,336.9 63,967.0 75.8 120.2
Net Private Net Returns 23,783.7 6,187.0 44.7 11.6
Benefits
Public Net Returns 1,831.6 3,461.8 3.4 6.5
Total Net Benefits 25,615.4 9,648.7 48.1 18.1
Ratios Private Returns to Investment 3.2 1.1 3.2 1.1
Public Returns to Investment 1.5 1.9 1.5 1.9
Net Returns to Investment 2.7 1.2 2.7 1.2
Private Investment to Public 2.9 13.3 2.9 13.3
Investment
Private Returns to Public 6.2 7.8 6.2 7.8
Returns
Estimates scaled up for 532 projects.
Analysis of the sensitivity of the base model indicates that the model is reasonably robust to individual changes in assumptions, with the exception of changes in private capital investment and (because some of the investments are fixed) number of projects.
Conclusion
This paper presents an initial attempt to develop a fuller and more
complete assessment of the costs and benefits derived from industrial modernization
and extension programs. It draws on data from the first year of activities
of the Georgia Manufacturing Extension Alliance (GMEA). Nonetheless, although
the results of the current study are provisional, they do indicate that
GMEA's industrial modernization resources are leveraging relatively high
levels of private investment which, in turn, are likely to lead to favorable
and positive public and private returns over time.
Acknowledgements
The authors acknowledge the valuable assistance of William Riall of
the Georgia Tech Economic Development Institute. They also benefitted from
comments provided by Irwin Feller, Dan Luria, Eric Oldsman, John Redman,
and Jeff Shacket. The authors are, however, solely responsible for the
content of this report.
References
Chrisman, James J. And W. Ed McMullan. "Static Economic Theory,
Empirical Evidence, and the Evaluation of Small Business Assistance Programs,"
Journal of Small Business Management. v. 40, April 1996, 56-66.
Feller, Irwin. Presentation at the NIST MEP Evaluation Working Group Meeting, Atlanta, GA, January 26, 1995.
Feller, Irwin and Gary Anderson. "A Benefit-Cost Approach to the Evaluation of State Technology Development Programs," Economic Development Quarterly 8, 1995, 127-140.
National Institute of Standards and Technology. Manufacturing Extension Partnership, electronic document available through http://www.nist.mep.gov (National Institute of Standards and Technology, U.S. Department of Commerce, Gaithersburg, MD), 1996.
Rosenfeld, Stuart. Competitive Manufacturing: New Strategies for Regional Development. Center for Urban Policy and Research Press, New Brunswick, NJ, 1992.
Shapira, Philip, J. David Roessner, and Richard Barke. "Public infrastructures for industrial modernization in the United States" Entrepreneurship and Regional Development 7, 1995, 63-84.
Shapira, Philip and Jan Youtie. "Assessing GMEA's Economic Impacts: Towards a Benefit-Cost Methodology." GMEA Evaluation Working Paper E9502. Georgia Institute of Technology, Atlanta, GA, 1995.
Shapira, Philip, Jan Youtie, and J. David Roessner, "Current Practices in the Evaluation of U.S. Industrial Modernization Programs," Research Policy, v. 27, n. 2, 1996.
Simons, Gene. Industrial extension and innovation, in L. M. Branscomb (Editor), Empowering Technology: Implementing a U.S. Strategy. MIT Press, Cambridge, MA, 1993.
Wood, William C. "Primary benefits, secondary benefits, and the evaluation of small business assistance programs" Journal of Small Business Management. v. 32, July 1994, 65-75.
The Georgia Tech Project on Industrial Modernization - www.prism.gatech.edu/~ps25/mod.htm
TOP OF THIS DOCUMENT | PUBLICATIONS MENU | PROJECT HOME PAGE