 |
Making Better Use of Business Survey Data: Thoughts on Overcoming the Anchoring and Nested Data Problems in Interpreting Business Survey Results
| |
| | Unformatted Document Text:
Edmund Malesky
19
Q39. Which Vietnamese province would you say is the most transparent? ........................................Which Vietnamese province would you say is the least transparent? ...................................... If the least transparent province received a rank of 1 and the best province a rank of 10, what score would your province receive?
27
Now, we are not guessing at what ideal form an enterprise was using as the shadow reference point for its province, we have clear anchors. The scale will be particularly helpful, if respondents identify the same provinces as endpoints, but even if they don’t, we have a much better understanding of their frame of reference. To aid firms, a list of both neighboring provinces as well as those well-known for advantageous and poor business environments was given. Firms needed only to choose from a set list of provinces.
Hopefully, firms that do not have business operations in other provinces may have enough information from newspapers and television to answer such a question. For as King et al point out, problems arise when firms do not have any information at all about their peers and must simply guess or answer that they don’t know. DIF cannot be eliminated when firms possess widely varying background levels of information and if such reference information variessystematically across provinces and not simply across firm size. Another problem with this technique is that its dependence on extremal statistics lowers statistical reliability.
28
Taken together, the two band-aids on business service data are not perfect but they advance our analysis far beyond our present survey instruments.
4. Nested data and the need for hierarchical modeling in analyzing BEEPS-type data.The second problem I encountered with business survey data arose when I begin experimenting with multivariate regressions and quickly realized the two standard approaches used by BEEPSauthors in their multiple papers would be severely limiting for my work.
29
The first technique is
to either use firms as the unit of analysis in the regressions and add country-dummies or country level variables as controls. The second is to take the mean response or the percentage of firms agreeing to particular questions as the dependent variable in provincial level regression.
The first technique is limiting because the firms are nested within provinces (and or states) and are undoubtedly influenced by unique characteristics of those provinces, but introducing provincial level characteristics offers no additional information, it is statistically equivalent to adding country-level dummies, because they do not vary along with firms. If one finds that these
28
King et al, 2003
29
Hellman, Joel, Geraint Jones, Daniel Kaufmann, 2000. “Are Foreign Investors and Multinationals Engaging in Corrupt
Practices in Transition Economies,” Transition, May-June-July. Hellman, Joel and Shankarman 2000, "Intervention and State Capture," EBRD Working Paper. Hellman, Joel, Geraint Jones, Daniel Kaufman, 2000. “Seize the State, Seize the Day: Sate Capture, Corruption, and Influence in Transition,” World Bank Policy Research Working Paper 2444, September.
|
| | Authors: Malesky, Edmund. |
|
| |
|
|
Edmund Malesky
19
Q39. Which Vietnamese province would you say is the most transparent? ........................................
Which Vietnamese province would you say is the least transparent? ......................................
If the least transparent province received a rank of 1 and the best province a rank of 10, what score would your
province receive?
27
Now, we are not guessing at what ideal form an enterprise was using as the shadow reference
point for its province, we have clear anchors. The scale will be particularly helpful, if
respondents identify the same provinces as endpoints, but even if they don’t, we have a much
better understanding of their frame of reference. To aid firms, a list of both neighboring
provinces as well as those well-known for advantageous and poor business environments was
given. Firms needed only to choose from a set list of provinces.
Hopefully, firms that do not have business operations in other provinces may have enough
information from newspapers and television to answer such a question. For as King et al point
out, problems arise when firms do not have any information at all about their peers and must
simply guess or answer that they don’t know. DIF cannot be eliminated when firms possess
widely varying background levels of information and if such reference information varies
systematically across provinces and not simply across firm size. Another problem with this
technique is that its dependence on extremal statistics lowers statistical reliability.
28
Taken together, the two band-aids on business service data are not perfect but they advance our
analysis far beyond our present survey instruments.
4. Nested data and the need for hierarchical modeling in analyzing BEEPS-type data.
The second problem I encountered with business survey data arose when I begin experimenting
with multivariate regressions and quickly realized the two standard approaches used by BEEPS
authors in their multiple papers would be severely limiting for my work.
29
The first technique is
to either use firms as the unit of analysis in the regressions and add country-dummies or country
level variables as controls. The second is to take the mean response or the percentage of firms
agreeing to particular questions as the dependent variable in provincial level regression.
The first technique is limiting because the firms are nested within provinces (and or states) and
are undoubtedly influenced by unique characteristics of those provinces, but introducing
provincial level characteristics offers no additional information, it is statistically equivalent to
adding country-level dummies, because they do not vary along with firms. If one finds that these
28
King et al, 2003
29
Hellman, Joel, Geraint Jones, Daniel Kaufmann, 2000. “Are Foreign Investors and Multinationals Engaging in Corrupt
Practices in Transition Economies,” Transition, May-June-July. Hellman, Joel and Shankarman 2000, "Intervention and State
Capture," EBRD Working Paper. Hellman, Joel, Geraint Jones, Daniel Kaufman, 2000. “Seize the State, Seize the Day: Sate
Capture, Corruption, and Influence in Transition,” World Bank Policy Research Working Paper 2444, September.
|
|
 Convention | | All Academic Convention is the premier solution for your association's abstract management solutions needs. | | Submission - Custom fields, multiple submission types, tracks, audio visual, multiple upload formats, automatic conversion to pdf. | | Review - Peer Review, Bulk reviewer assignment, bulk emails, ranking, z-score statistics, and multiple worksheets! | | Reports - Many standard and custom reports generated while you wait. Print programs with participant indexes, event grids, and more! | | Scheduling - Flexible and convenient grid scheduling within rooms and buildings. Conflict checking and advanced filtering. | | Communication - Bulk email tools to help your administrators send reminders and responses. Use form letters, a message center, and much more! | | Management - Search tools, duplicate people management, editing tools, submission transfers, many tools to manage a variety of conference management headaches! | | Click here for more information. |
|
|
|
| |
|
|
|
