NOTE NUMBER 352
viewpoint PUBLIC POLICY FOR THE PRIVATE SECTOR
MARCH 2018
Trade Facilitation Reform
MACROECONOMICS, TRADE & INVESTMENT GLOBAL PRACTICE
Russell Hillberry, Ana The Impact of Targeted Sampling on Import Compliance
Margarida Fernandes,
and Alejandra Mendoza
Alcántara The re sults of a n im p a c t e v a lua t io n o f t he int r o d uc t io n o f a r i s k -
Russell Hillberry b ase d ap p roach t o s a m p ling im p o r t s hip m e nt s b y t he Ma c e d o n i a n
(rhillber@purdue.edu)
is an Associate Profes- Food and V eteri na r y Ag e nc y ind ic a t e t ha t t he s a m p ling p la n w a s
sor in the Department of
Agricultural Economics at
not e f f e ctive ly i m p le m e nt e d , a nd t he r e f o r m d id no t im p r o v e t h e
Purdue University. Ana targe ting of risk y s hip m e nt s . Ev id e nc e t ha t t he r e f o r m inc r ea s e d
Margarida Fernandes
(afernandes@worldbank. trad e f lows is inc o nc lus iv e . T he we a k r e s ult s s ug g e s t a ne e d t o
org) is a Senior Economist
in the Trade and Interna- conce ntrate ef f o r t s in im p r o v ing d a t a m a na g e m e nt in t e c hn i c a l
tional Integration Unit of
the Development Research age ncies, not only f o r a na ly t ic a l p ur p o s e s b ut e s p e c ia lly t o i m p r o v e
Group at the World Bank.
Alejandra Mendoza
the d esign and e f f e c t iv e ne s s o f t he r e f o r m s .
Alcántara (amendoza1@
worldbank.org) is an Im-
pact Evaluation Specialist Every year hundreds of millions of dollars of former Yugoslav Republic of Macedonia (FYR
in the Finance, Competi-
development aid are spent on trade facilitation Macedonia).
tiveness and Innovation
Global Practice at the projects. Few projects are subjected to scrutiny via Prior to the reform, the World Bank Group’s
World Bank Group. formal impact evaluations, and most of the stud- Western Balkans Trade Facilitation project iden-
Research for this note was ies that have been conducted evaluate reforms tified lengthy and intrusive sampling and labora-
supported by the govern- by customs agencies.1 tory testing regimes by technical agencies as a
ments of Canada, the This note is part of a research series on trade significant constraint on trade in FYR Macedonia.
United Kingdom, and
the United States through facilitation reforms in the Western Balkans. It Laboratory tests of sampled shipments very rarely
THE WORLD BANK GROUP
the Impact Program of is the first study to evaluate a trade facilitation identified noncompliant shipments, suggesting
the World Bank Group’s reform undertaken by a “technical agency” (an poor targeting and excessive sampling. In 2013,
Trade and Competitive-
ness Global Practice.
agency tasked with oversight of food safety, for example, samples were taken from nearly 1 in
Additional support was pro- environmental protection, or other respon- 10 import shipments. Less than 0.2 percent of the
vided by the World Bank’s sibilities beyond the remit of the customs samples taken were found to be noncompliant
Multi-Donor Trust Fund
agency). The study evaluated the introduction (authors’ calculations based on FVA administra-
for Trade and Develop-
ment, the Strategic Research of a risk-based approach to sampling of import tive data for imports of food of animal origin
(continued on back page) shipments for the purpose of laboratory testing and animal feeds). High rates of sampling sub-
(Fernandes, Hillberry, and Mendoza 2017). jected imports to queuing at the border, causing
The reform was undertaken in 2014 by the delays during the movement of goods. Shippers
Food and Veterinary Authority (FVA) of the of goods that were sampled for testing also had
AR
T AR
TT DA TA
ECF CI
IN GL IFTD IO
AIT H RUECF
N M
OW HO D
ROMESTH
IN IM
EV E PM
ST CN
AE T TOC
F L IT M
AART
GE EA
E TM DT T
SEAR
M ?P L I N G O N I M P O R T C O M P L I A N C E
to wait for laboratory testing results before their sampling activity match the sampling activity
goods could be released to the market. planned in the reform? Did the reform cause
The research reviewed here used data on sampling activity to become better linked to the
planned and actual sampling activity, labora- probability that samples do not comply with the
tory test outcomes, and international trade flows FVA’s technical standards? How did the reform
to assess the impact of the reform. It provides affect import flows?
answers to three questions: How closely did actual
Improving sampling procedures in the FVA
The goal of the reform was to reduce administra-
2 Planned and actual number of import shipments sampled by the Food and tive and private sector costs through adoption of
Figure Veterinary Authority, by product and country group, 2014 and 2015
1
a risk-based approach to sampling import ship-
a. 2014 ments for laboratory testing. With World Bank
Group support, in 2013 FVA staff received train-
600 Actual
ing on risk-based approaches. Training included
Planned
the development of risk scores and guidance on
constructing an annual sampling (or monitor-
Number of Samples
400
ing) plan based on historical data. The objective
of this process was to enable the FVA to better
focus sampling and testing efforts on shipments
200
that were more likely to be noncompliant with
technical standards and more dangerous if
noncompliant. More focused sampling activity
0 would improve oversight of import shipments
w EF A
t – – C tic
pr ne ica
& – & A
M EF U & TA
Pe NA FTA
– C SA
– A Fis FTA
rk oo SA
U rk – A
C NA
er )
ll
& ll
)
& All
xc ultr ts – l
U B ll
E il
try try ll
M A ll
– E eat )
E ll
nd
A
itz TA
l
d sh TA
ne M FTA
– S & FT
U ilk U FT
T
vi U & A
A
ll Po duc – A
. E y– A
& raz
ul ul A
–N –A
U –A
ed M FT
while reducing the number of shipments sent
Pe ed Bal
C E EF
c. P EF
sh Ho Afr
la
Bo – E ilk –
–E d–
si h –
Po Po sc –
ilk &
Po t f &
un Fi EF
ilk TA E
.E M E E
– E
E
o y
fe –
C
&
o
lu
U
&
a
for laboratory testing. The sampling plan out-
ol
ne
M
E
sh
fe
lined the number of samples to be taken the
i
ex
v
Fi
(e
Fi
Bo
po
d
l(
un
Al
om
–A
po
following year for a given border post, product
l–
xc
C
ne
om
ffa
(e
vi
O
ll
C
Bo
–A
group, and source country or group of countries.
ed
fe
Sampling plans were designed for three sets of
d
un
po
products under FVA oversight: food of animal
om
C
b. 2015 origin, animal feeds, and food of nonanimal
Actual
origin and food contact materials.
1,000
Planned
800 Weak adherence to sampling plan
Number of Samples
The annual sampling plans that were designed as
600 part of the reform aim to optimize sampling activ-
ity by border agents, taking into account expected
400 probabilities of noncompliance and potential
degrees of harm, given resource constraints.
200
Each annual plan is intended to guide sampling
activity over the subsequent year, although agents
0
can deviate from the sampling plan as necessary.
a
tic
ea SA
rk A
fe FTA
ul U & TA
A A
ey )
C il)
Po sh – ll
A l
ll
try l
xc us All
p ffa ll
– B il
& ll
un & ll
C ffal uc All
– C ll
si k – l
SA
Al
ul Al
– A Mil Al
ne fric
Po FT
– N EFT
Fi A
–A
O –A
z
A A
po EU – A
ilk – A
z
al
Po – E EF
Fi Bra
ra
–
ll Mo d –
. N c–
O rod l –
–
M &
E
E
The discovery of salmonella in products imported
om – ts
t
ed
EU B
oo
Bo &
–
un lk – xc.
on
&
sh
a
(e ll
tf
H
d
po Mi l (e
try
vi
into another country in the European Union, for
Pe
M
ed
Al
sh
sh
fe
–
–A
Fi
ne
Fi
example, would likely generate additional FVA
d
vi
try
Bo
ul
sampling activity on similar shipments. Even
Po
om
C
Source: Fernandes, Hillberry, and Alcantara 2017.
Note: Figures show information based on sampling plans; the FVA laboratory samples dataset; and the single window for import,
absent news of a bacteria outbreak, FVA border
export, and transit of goods and tariff quota (EXIM) dataset. The datasets are combined based on the broad product group and cover agents retain discretion to depart from the sam-
food of animal origin and animal feeds.
Group of countries: All countries (All), North America (NA), South America (SA), European Union (EU), Central European Free Trade pling plan when they suspect that a shipment
Agreement members (CEFTA), European Free Trade Association members (EFTA).
warrants additional scrutiny.
Raw summary statistics reveal sizable differ- results. Oversampling in 2014 can be attributed
ences between actual sampling activity and the to the fact that implementation of the reform
sampling plans. The sampling plan for 2014 pro- was delayed until July of 2014. Actual sampling
jected that 598 samples would be taken from two rates in 2014 therefore included six months of
classes of products: food of animal origin and sampling activity that preceded the plan. In 2015,
animal feeds. The actual number of samples taken a funding dispute between the FVA and the labo-
surpassed the plan by a factor of three (figure 1a). ratories that test the samples dramatically limited
In contrast, in 2015 the actual number of samples sampling activity.
taken was 23 times smaller than the planned num-
ber (figure 1b). FVA failed to target risky shipments
A regression model was used to assess the cor- One of the objectives of the risk-based sampling
relation between the actual samples taken in 2014 reform was to align sampling activity with the
and 2015 and the corresponding sampling plans.2 likelihood that shipments would be judged non-
The analysis was limited to the two sets of products compliant by subsequent laboratory tests. If the
subject to FVA oversight for which complete data reform was effective, a stronger positive relation-
are available: food of animal origin and animal ship between noncompliance rates and sampling
feeds. The regression controlled for the inherent rates would be observed after the reform than
risk characteristics associated with each group of before it. A regression model was used to test this
products, group of source countries, and year.3 effect.4 The estimation controlled for unrelated
The results indicate weak adherence to the factors, such as differences in product character-
sampling plan by FVA border agents. A posi- istics or the potential for harm.5
tive coefficient found on the planned number A preliminary analysis showed that the num-
of samples suggests that deviations from the ber of shipments that did not comply with
sampling plan leaned in the direction of over- the FVA’s technical standards in 2013–15 was
sampling shipments that had initially been extremely small (fewer than 40). In the second
deemed most risky (those with higher numbers semester of 2014, sampling and noncompliance
of planned samples). rates increased for food of animal origin, possibly
Two unexpected challenges to implemen- because of the 2013 aflatoxin outbreak (table 1).
tation of the reform plan help explain these Given the very small number of noncompliant
Table Sampling and noncompliance rates of all imports, 2013–15
1
Number of sampled Non-compliance
Number of import Total number of import shipments Sampling rate rate
Year shipments sampled import shipments noncompliant (percent) (percent)
2013
First half 835 8,167 12 10.3 1.4
Second half 446 8,479 1 5.3 0.2
Total 1,281 16,646 13 8 1
2014
First half 451 7,932 2 5.7 0.4
Second half 780 8,643 21 9.1 2.7
Total 1,231 16,575 23 7 2
2015
First half 4 8,844 0 0 0
Second half 59 9,615 0 0.6 0
Total 63 18,459 0 0 0
Source: Fernandes, Hillberry, and Alcantara 2017.
Note: Table shows information based on the FVA laboratory samples dataset and the EXIM dataset. The datasets are combined using the broadly defined product
grouping and cover food of animal origin and animal feeds.
shipments, the regression estimates were unable for FVA products that coincided with the reform
to identify a significant relationship between the (though efforts were undertaken to control for
likelihood of noncompliance and the sampling export supply and import demand shocks and to
rate before the 2014 reform. Considering also restrict the sample of control products to be more
data from the post-reform period, the analysis “similar” to the FVA products to rule out this
reveals that the probability of a shipment being possibility). If the estimates of an additional 3–5
noncompliant was lower when sampling rates percent growth in imports of FVA products are
were higher, especially after the reform. correct, trade response parameters suggest that
This result is concerning, because risk-based the reform had an impact roughly equivalent to
4
targeting aims to focus sampling activities on ship- a 1-percentage point tariff cut on FVA products.
ments that are more likely to be noncompliant. The second exercise used a regression model
One would therefore expect the probability of to assess the relationship between changes in the
noncompliance among sampled shipments to sampling rate and changes in import value. This
be higher when sampling rates are higher.6 The exercise was complicated by the 2015 funding
findings suggest that the targeting of sampling dispute with the laboratories, which caused a dra-
activity associated with imports of food of animal matic drop in the sampling rate that is not attrib-
origin and animal feeds was not effective in FYR uted to the reform. The relationship between
Macedonia. sampling rates and import value is of interest to
A caveat with these and other exercises on the trade facilitation literature, so it was estimated
sampling activity is that the data needed to calcu- with and without the 2015 data, which include
late sampling rates for less risky products (food the effects of the funding dispute. The exercise
of nonanimal origin and food contact materials) found no evidence of an effect of reduced sam-
were not collected. Given that those products pling on import value over the 12–18 months
are less likely to pose health risks to the public immediately after the reform.
if noncompliant, the FVA reform was designed Overall the estimates indicate inconclusive
to reduce their sampling rates by larger magni- effects of reduced sampling rates on import value.
tudes. The reform may have succeeded in ways Changes in the sampling rate had no immediate
that could not be measured directly. effect on import value; it is possible that such
effects emerge with a lag over longer time hori-
Limited impact of reform on trade zons. The findings reveal that imports of FVA
A central question typically asked in the small products grew slightly faster than various sets of
literature evaluating trade facilitation reforms is control products following the reform, though it
the degree to which such reforms affect trade. In is not possible to say conclusively that this growth
this study, this question was addressed in two ways. was a result of the reform. These findings need
First, an exercise examined whether imports of to be interpreted in light of the caveat that it was
products subject to FVA oversight saw more rapid not possible to estimate relationships between
growth than other products following reform. sampling rates and imports for less risky products,
Second, an exercise tested whether larger reduc- for which larger changes in the sampling rates
tions in the sampling rate led to larger increases were planned.
in the import value of products subject to FVA
oversight. Technical agencies need to improve data
In the first exercise, a difference-in-difference collection efforts
regression was used to compare import growth of This research highlights the challenges asso-
products subject to FVA oversight with growth of ciated with an impact evaluation of reforms
similar products. It found that the import value undertaken by technical agencies that play an
of FVA products rose 3–5 percent following the important role at the border but are unable to
reform. These estimates include the effects of any collect comprehensive data on variables of inter-
other FVA policy changes implemented at the est. One reason data are lacking is that techni-
same time as the reform. They may also be con- cal agencies sometimes collect data in ways that
flated with unobservable changes in the market are not consistent with the international trade
classification system. A lack of comprehensive Conclusion
data on test outcomes also limited the evalua- This study is innovative for three reasons: (a)
tion of substantial reforms that occurred in the it applies rigorous impact evaluation methods
sets of products imposing smaller public health to trade facilitation reforms; b) it explores the
risks. In the FVA, these data challenges not only impact of the reform on compliance with import
undermine evaluation efforts, they also limit the regulations, an outcome the literature previously
effectiveness of both the risk-based system and had ignored; and c) it assesses for the first time
the construction of the sampling plans. a reform implemented by a technical agency.
Recommendations for improving data man- The study highlights the need to concentrate
5
agement include: efforts in improving data management in such
agencies, not only for analytical purposes but
1. The FVA should itself collect all test out- particularly to improve the effectiveness and
comes—positive and negative—in an elec- design of subsequent reforms. In this case, a
tronic format or platform, rather than relying data management reform would help track the
on the laboratories to collect and provide this effectiveness of the sampling plan, monitor its
information. The information collected from implementation, and improve targeting based
each import shipment sampled should be on shipment characteristics.
linked to the shipment identification number Many of the study’s findings are inconclusive,
issued by the single window for import, export, because of the short sample period; the weak
and transit of goods and tariff quota (EXIM), quality of the data; and specific implementation
which collects information on the volume of issues, such as the funding dispute with the public
imported products, the country of origin, and laboratories in 2015. But the research highlights
the firm engaged in each import transaction. the fact that several underlying mechanisms—
Electronic data collection would allow more including the design of the sampling plan, the
precise tracking of the characteristics of non- testing process, and training for border agents—
compliant shipments and the updating of the make trade facilitation reforms work. The con-
sampling plan in real time. tributions of these subcomponents represent
knowledge gaps that could be filled by a different
2. A more ambitious data management reform evaluation design.
would seek to integrate information from (See back page for Notes and References.)
laboratory test results with data recorded by
other agencies, especially the customs agency.
Data at the shipment level could be used to
improve the targeting strategy and the sam-
pling plan. Incorporating shipment-level data
contained in the customs declaration would
require high levels of coordination with the
national customs agency.
3. Sampling plans should be updated more than
once a year and take into account emerging
reports of outbreaks or evidence of failed
outcomes at other agencies. Emerging risks
are currently communicated to border agents
through other channels; updating the sam-
pling plan accordingly makes sense. Frequent
updating of the sampling plan would better
tie actual sampling behavior to the plan and
help the agency learn about the effectiveness
of its approaches to updating the plan.
TRADE FACILITATION REFORM THE IMPACT OF TARGETED SAMPLING ON IMPORT COMPLIANCE
Notes References
1. See, for example, Fernandes, Hillberry, and Alcan- Carballo, Jerónimo, Georg Schaur, Alejandro Graziano,
tara (2015); Fernandes, Hillberry, and Berg (2016); and Christian Volpe Martincus. 2016. “Transit
Carballo, Schaur, Graziano, and Martincus (2016); and Trade.” IDB Working Paper IDB-WP-704, Inter-
viewpoint
Carballo, Schaur, and Martincus (2016). American Development Bank, Washington, DC.
2. The regression model estimated by ordinary least Carballo, Jerónimo, Georg Schaur, and Christian Volpe
is an open forum to
squares (OLS) explains the relationship between the Martincus. 2016. “Trust No One? Security and Inter-
encourage dissemination of
number of import shipments sampled and the number national Trade.” IDB Working Paper IDB-WP-703,
public policy innovations
of samples to be taken according to the sampling plan, Inter-American Development Bank, Washington,
for private sector–led and
at the product-group of country-year level. DC.
market-based solutions for
3. This control was handled by including fixed effects Fernandes, Ana M., Russell H. Hillberry, and Alejandra
development. The views
for the product group, the country group, and the year Mendoza Alcantara. 2015. “Trade Effects of Customs published are those of the
as regressors in some of the OLS regressions. The as- Reform: Evidence from Albania.” Policy Research authors and should not be
sessment also excluded outliers, such as milk and milk Working Paper 7210, World Bank, Washington, DC. attributed to the World
products, for which Europe set up a warning in 2013 ———. 2017. “An Evaluation of Border Management Bank or any other affiliated
following the outbreak of aflatoxin, and pet food prod- Reforms in a Technical Agency.” Policy Research organizations. Nor do any
ucts, for which the number of shipments to be sampled Working Paper 8208, World Bank, Washington, DC. of the conclusions represent
increased dramatically in 2015. Fernandes, Ana M., Russell H. Hillberry, and Claudia N. official policy of the World
4. The regression model estimates a relationship Berg. 2016. “Expediting Trade: Impact Evaluation of Bank or of its Executive
between an indicator for noncompliance of an import an In-House Clearance Program.” Policy Research Directors or the countries
shipment and the associated sampling rate. Sampling Working Paper 7708, World Bank, Washington, DC. they represent.
rates are calculated as the ratio between the number of
shipments sampled and the total number of imported To order additional copies
shipments for each broad product group and semester. contact Jenny Datoo,
5. This control was handled by including in the OLS managing editor,
regressions fixed effects for the product group, the Room F 5P-504,
The World Bank,
country group, and the semester-year.
1818 H Street, NW,
6. The sampling and noncompliance rates were nega-
Washington, DC 20433.
tively correlated even in the second semester of 2014,
when the funding dispute with the laboratories, which
Telephone:
undermined the analysis of 2015, was not a factor.
001 202 473 6649
Email:
jdatoo@worldbank.org
Produced by Carol Siegel
Printed on recycled paper
(continued from front page)
Program on Economic
Development, and the Impact
Evaluation for Development
Impact (i2i) Fund.