INCORPORATING MANAGED PREFERENCES IN THE EVALUATION OF PUBLIC ORGANIZATIONS EFFICIENCY: A DEA APPROACH

Tiago Silveira Gontijo

Izabela Hendrix University, Brazil

E-mail: tsgontijo@hotmail.com

Alexandre de Cássio Rodrigues

FUMEC University, Brazil

Izabela Hendrix University, Brazil

E-mail: alexandrerodrigues.engprod@gmail.com

Cristiana Fernandes De Muylder

FUMEC University, Brazil

E-mail: cristiana.muylder@fumec.br

Submission: 18/07/2017

Revision: 04/04/2018

Accept: 16/04/2018

ABSTRACT

The classical Data Envelopment Analysis (DEA) allows us to evaluate public organizations’ effectiveness, however, such models can classify as efficient organizations that, in fact, are not. This paper aims to evaluate, through a DEA model that incorporates managerial preferences, the efficiency of the twenty five National Department of Mineral Production (DNPM) superintendence's. In this paper we considered as output the amount of servers in the middle and end areas. The number of mining titles granted in 2016 was analyzed as input. In order to upgrade the classical DEA mode, we utilized manager preferences regarding outputs by the assurance region method. The results exhibited that, when incorporating management preferences into the classic DEA models, the superintendence number that showed maximum operational efficiency reduced from eight to five. For superintendence classified as inefficient, we identified the benchmarks and a performance target, since they can support the action planning aimed at reducing the high liabilities pending processes for analysis by the municipality. The findings of this study shall reduce the DNPM's slowness in granting mining bonds, contributing to the Brazilian economy.

Keywords: efficiency; managerial preferences; data envelopment analysis

1. INTRODUCTION

The New Public Management movement, which emerged in the 1970s, presented itself with the primary objective of "transform the government" into a private company, thereby acquiring efficiency, reducing costs, and achieving greater productivity in service delivery (MOTTA, 2013). As a consequence, it has become common for public organizations to use increasingly sophisticated performance evaluation systems (NOGUEIRA et al., 2012).

In the Brazilian context, the inclusion in 1998 of the efficiency principles in the Public Administration stimulated the applied academic research, with the purpose of measuring the government efficiency (DINIZ; LIMA, 2014) using classical Data Envelopment Analysis (DEA) models (MACEDO; NOVA; ALMEIDA, 2010). The DEA is a benchmarking technique, since it compares similar operating units, which spent inputs to produce outputs, (LIU et al, 2013; EMROUZNEJAD; YANG, 2018).

A major limitation of classical DEA models is the flexibility in the selection of weights to assign to the choiced output / input in determining the efficiency of each Decision Making Unity (DMUs) (Charnes, Cooper 1978; Banker, Charnes, & Cooper, 1984), that is, the classical models do not incorporate information on the managers' preferences (LIU; SHARP; WU, 2006; THANASSOULIS; PORTELA; ALLEN, 2004). This flexibility makes the method benevolent, (ROLF; SHAWNA; DIMITRIS, 2015; ZHU, 2015), enabling a bias in the analysis (FERREIRA; GOMES, 2009). This limitation strongly affects the public organizations efficiency evaluation. After all, a public school with high approval level but low learning target can be classified as efficient, although these "two concepts are equally important for the education quality" (INEP, 2017).

When stakeholders have managerial preferences, it is possible to incorporate the preferences into DEA models by new constraints that deal with the appropriated weights to inputs and outputs. Joro and Korhonen (2015) presented a global review of the DEA models evolution, including the weight restriction methodology. Recent applications of these models include the efficiency evaluation of public hospitals (GONÇALVES, 2010), participating countries of Olympic Games (SHIROUYEZZAD; YAZDANI, 2014; LI et al., 2015), electric power companies (SARTORI, 2016), and graduate programs (SILVA; CORRÊA; GOMES, 2017).

This paper aims to apply DEA models with preferences to evaluate the efficiency of the National Department of Mineral Production (DNPM) superintendencies, a federal authority responsible for promoting the granting of mining titles. The DNPM's "slowness" in granting mining titles, a possible consequence of the low labor force and the high liabilities of pending review processes (TCU, 2011; DNPM, 2017c) has resulted in a reduction in the Investment in the mineral sector (TOMAZ, 2014), which is strategic for the Brazilian economy (IBRAM, 2015). The choice of DEA models with preferences is justified because DNPM managers, depending on the goals they have to fulfill to be entitled to a performance bonus (BRASIL, 2004), have different preferences on the titles granted.

2. DEA

2.1. Classic DEA models

Efficiency is a relative concept that compares what has been produced, given the available resources, with what could be produced with the same resources (ZHU, 2015). Efficiency makes use of two approaches which vary according to the production process, as follows: (i) orientation to inputs (minimize the use of inputs given level of output); (ii) orientation to outputs (maximize the level of output given levels of the inputs) (COOPER; SEIFORD; ZHU, 2011).

The DEA is a non-parametric efficiency measurement technique, which was disseminated especially from the seminal works of Charnes, Cooper, and Rhodes (1978) and Banker, Charnes, and Cooper (1984). The main difference between these two works is that, while the first presupposes constant returns to scale (RCE), that is, any variation in the inputs implies proportional variation in the outputs, the second involves the assumption of variable returns to scale (RVE).

In the DEA models’ mathematic formulation, it is assumed that N Decision Making Units – DMUs utilize the same production technology to transform m inputs in s outputs. Therefore, the efficiency score of the DMU₀ (object),, is given by:

(1)

where (i = 1,..., m) and (j= 1, ..., s) detonate the weights that the DMU_o give to the inputs and outputs, respectively. The conditions can be formalized as follows.

(2)

Although it allows us to easily interpret the efficiency of a DMU, the problem expressed in (2) admits infinite solutions (ZHU, 2014). To circumvent this situation in the output-oriented models, which will be used in this work, one must make the numerator of the objective function equal to a constant, usually one, and transform the constraint into a difference between the numerator and the denominator, which makes the efficiency scores between zero and one. The result is the multiplier models, which are shown in Table 1:

Table 1: DEA models, under RCE and RVE, with orientation to outputs

DEA/RCE Model		DEA/RVE Model
	(3)		(4)

Source: Adapted from ZHU (2014, p. 50).

As expressed in the problem (4), it is an unrestricted variable in signal that indicates whether scale return is constant (= 0) or variable (≠ 0). Denoting the optimal solution of (3) and (4) by, the DMU_o will be efficient, if and only if, =1 and all of the e values are positive. Otherwise, the DMU_o will be classified as inefficient one, and its benchmarks will be the DMUs associated with the active inequality constraints in the optimal solution:

(5)

The efficiency score of DMU_o calculated under the constant returns to scale assumption,, measures the overall technical efficiency, while that obtained under the variable returns to scale assumption, , Measures the pure technical efficiency, which is related to the operational aspect. The ratio between these measures provides the efficiency to scale, which indicates that the DMU operates at an optimal scale (ZHU, 2014).

2.2. DEA MODELS WITH PREFERENCES

The classical DEA models attribute weights to the inputs and the outputs, so they maximize the DMUs efficiency scores. This allows to identify inefficient DMUs, which perform poorly even after choosing the weights that are most favorable to them. In addition, such flexibility ignores any preferences of managers in relation to inputs and outputs, since larger weights can be attributed to minor variables, which makes an a priori inefficient DMU Classified as efficient.

Managerial preferences may include judgments about prior views regarding inputs and outputs, the relationship between some inputs and outputs, efficient and inefficient DMUs, and input/output substitutions (Allen et al., 1997). These preferences can be incorporated into the classic DEA models by restricting the weights assigned to inputs and outputs. In this sense, the methods of direct restriction to weights, safety regions, cone ratio and restriction to virtual inputs and outputs are highlighted.

The direct weight restriction method, generalized by Roll et al. (1991) imposes limits on multipliers for the purpose of not ignoring or overestimating inputs and/or outputs in the analysis. These constraints are given by (6), where V and U are constants that respectively represent the limits imposed on the inputs and outputs weights. A drawback of this method is the possibility of generating an infeasible linear programming problem.

(6)

The assurance region method, proposed by Thompson et al. (1990), adds limits to the multipliers, thus restricting weights to a given region. For this, the following restrictions on input and output weights are added to the classical DEA models, respectively:

(7)

From each of the presented constraints in (7), we derive two more conditions, which are given by (8) and (9), respectively:

	(8)
	(9)

The cone ratio method is a generalization of the safety region method by Charnes et al. (1989). In this approach, the weights assigned to the inputs are constrained by a convex cone defined by k vectors (i = 1, ...., k):

(10)

Similarly, the weights assigned to the outputs are constrained by a connected cone defined by l vectors (j = 1, ...., l):

(11)

The method proposed by Wong and Beasley (1990) imposes a limitation on the proportion of the total virtual input of the DMU_o used by input i (output j) to the interval , stipulated by the decision maker, which reflects the importance given to input i (output j) by DMU_o. However, this limitation, expressed in (12), can lead to problems of infeasibility of difficult solution (ALCÂNTRA; SANT'ANA; LINS, 2003).

(12)

3. MATERIAL AND METHODS

The methodological instrument used was an ex post evaluation, classified as a cross-sectional work based on quantitative methods (DEA, in particular). According to the most recent information available the data, related to the 2016 year, were obtained on the site and upon request to the Citizen Information Service of the Municipality. Considering that the DEA technique aims to compare the efficiencies of productive units that perform similar activities, such as DMUs, DNPM’s superintendence:

I - carry outs activities related to levy, charging, granting, surveys, citizen-user assistance, fiscal action, legality analysis of , obtaining data and information on mineral economy and the use of geotechnologies;

II - promotes budgetary and financial execution within its constituency; and

III - make materials management, assets, documents, personnel, infrastructure, information technology and general services (MME, 2011, Art. 85).

The sample consisted of all the DNPM superintendence in the federation states (DMUs), with the exception of Acre. However, the circumscription of the supervisory authority of DNPM (Goiás) covers the Federal District and the oversight of the DNPM (Rondônia) covers Acre. Thus, the research included 25 superintendence. Once defined the DMUs, inputs/outputs were selected. This is a fundamental step in DEA, since the efficiency scores are directly influenced by these variables (COOK; TONE; ZHU, 2014). There are no previous studies on the DNPM efficiency, so the inputs/outputs used in recent works that evaluated the efficiency of Judiciary were used as reference (Table 2). This is consistent because the DNPM, like the Judiciary, deals with process analysis.

Table 2: Inputs and outputs of judicial branch organizations

Work	Inputs	Outputs
Nogueira et al. (2012)	Total expenditure, total staff, IT expenses, new cases, total magistrates and internal resources.	Costs, recollections and several sentences.
Diniz and Lima (2014)	Total expenditure.	Number of cases dropped in 1st and 2nd grades, in the special court and in the recursal class.
Araújo, Dias and Gomes (2015)	Number of pending and new cases and number of servers awarded and assigned.	Number of sentenced and resolved cases.
Oliveira et al. (2016)	Number of non-criminal and criminal cases and misdemeanors, total employees number and number of computers.	Number of dispatches, returns, judgments, hearings, interlocutory decisions, actions, conciliations and cumulative activities.

Source: Prepared by the authors based on Nogueira et al. (2012), Diniz e Lima (2012), Aráujo, Dias e Gomes (2015) e Oliveira et al. (2016).

As was reported in Table 2, it is possible to observe that, in general, the inputs are associated with the number of servers and the outputs, with the number of analyzed processes. In the DNPM case, these variables are quite appropriate, since the low labor force of the municipality and the high liabilities of pending analysis processes (TCU, 2011; DNPM, 2017) make problems in mining titles granting, which, consequently, has hindered the institution mission fulfillment. Therefore, we consider the inputs and outputs as follows (Table 3):

Table 3: Inputs and outputs from DNPM superintendencies

Inputs
Indicator		Relevance
1. Number of effective servers in the middle area (serv_middle)	The DNPM middle area is made up of servers with assignments geared to the administrative exercise and logistic activities, which make use of all the equipment and resources available for the accomplishment of these activities.
2. Number of effective servers in the end area (serv_end)	The DNPM final area consists of servers with attributions dedicated to the activities inherent to the promotion and control of the exploration and mineral resources exploitation, to the inspection and fossiliferous deposits protection, to the monitoring and analysis of geological, mineral and mineral technology research, The performance monitoring of the Brazilian and international mineral economy, the mineral policy implementation, the promotion of the rational and efficient mineral resources use, supervision of the collection of financial compensation for the mineral resources exploration, and fostering the development of scientific and technological research, aimed at knowledge, sustainable use, conservation and mineral resources management.
Outputs
Indicator	Relevance
1. Number of published research permits (perm_pub)	The Research Permit authorizes the execution of works focused on the deposit definition, its evaluation and the determination of the feasibility of its economic use.
2. Number of licenses granted (lic_granted)	Licensing accredits its possessor the mineral exploitation of substances destined for immediate employment in construction.
3. Number of mining permits granted (perm_granted)	The mining permit allows the use of mineral extractable materials, which by their nature, especially its small volume and irregular distribution, do not often justify investment in research work.
4. Number of extraction records granted (reg_granted)	The extraction register allows to the organs of the direct or autarchic administration of the Union, the States, the Federal District and the Municipalities, exclusively, to extract substances of immediate use in the civil construction, so that they are used only in public works, being prohibited its sale, Third party or transfer to private companies.

Source: Elaborated by the authors based on avaliable information in Brazil (2004) and DNPM (2017b).

The indicators related to the inputs were obtained by request to the Citizen Information Service of the municipality (DNPM, 2017d) in June 2017, as these were not detailed in the Management Report of 2016 (DNPM, 2017c). The outputs were collected on the entity's website (DNPM, 2017a). The descriptive statistics of the selected inputs and outputs are shown in Table 4:

Table 4: Descriptive statistics of selected inputs and outputs

Variable	Type	Minimum	Maximum	Mean	Stan. deviation
serv_middle	Input	2	31	13,32	7,34
serv_end		2	58	17,12	13,40
perm_pub	Output	15	2926	539,36	734,74
lic_granted		7	251	66,52	62,09
perm_granted		0	150	7,84	29,82
reg_granted		0	45	5,96	11,68

Source: Prepared by the authors based on data from DNPM (2017d) and DNPM (2017a)

Table 4 highlights that at least one of the outputs related to the number of mining permits granted (perm_granted) and the number of extraction records granted (reg_granted) was null. Since the DEA models only admit non-zero positive variables (CHARNES; COOPER; RHODES, 1978; BANKER; CHARNES; COOPER, 1984), for all DMUs a unit was added to those outputs. This procedure, consisted on variables translation, which does not change the efficiency scores by DEA, since it moves the efficiency frontier for all DMUs (ZHU, 2015).

It is worthwhile to note that the large inputs and outputs amount compared to the number of DMUs decreases the discriminating power of the DEA (COOK; TONE; ZHU, 2014). In this sense, Ferreira and Gomes (2009, p.149) recommend that "for each pair of input variables and for each pair of product variables [one should be excluded] when they have high correlation (for example, above 0.8) ". Thus, to verify that condition, we calculated the correlations between the variables, which are shown in Table 5.

Table 5: Correlation between variables

Variable	serv_middle	serv_end	perm_pub	lic_granted	perm_granted	reg_granted
serv_middle	1,00
serv_end	0,62	1,00
perm_pub	0,47*	0,80	1,00
lic_granted	0,47*	0,83	0,86*	1,00
perm_granted	0,19	0,32	0,32	0,36	1,00
reg_granted	0,29	0,58	0,76	0,60	0,35	1,00

Source: Prepared by the authors based on data from DNPM (2017d) and DNPM (2017a) Note: The asterisk indicates significant correlation at the 5% level (2 tailed Pearson correlation test).

Table 5 shows that the correlation between the inputs (0.62) was less than 0.8 and not significant at the 5% level. Therefore, through the correlation criterion, it was not possible to exclude any input. In terms of the outputs, it was noted that lic_granted and perm_pub had high correlation (0.86), which was significant at the 5% level. However, it was decided to maintain the outputs, since those variables being tied to the evaluation of the institutional performance of the DNPM, as will be shown below, the discrimination power of the DEA is little affected when the number of DMUs is equal to a minimum of three times the amount of inputs and outputs (ZHU, 2015). Table 5 shows that, except for service with perm_pub and lic_granted, no significant correlation was observed between inputs and outputs, which is not an impediment because the DEA technique does not require a functional relational between inputs and outputs (FERREIRA; GOMES, 2009).

In the data analysis, output-oriented DEA models were adopted. The DNPM superintendencies were classified as efficient, taking into account if the number of servers in the middle and end areas can maximize the number of research permits publications and licensing concessions, mining permit mining and extraction logs. The orientation to outputs was appropriate since the orientation to inputs was adopted, and the objective would be to reduce them, maintaining the current levels of outputs.

It is important to note that it is desirable to increase the outputs, since, according to an audit carried out by the Federal Audit Court (TCU), the liabilities of DNPM's pending cases are high (TCU, 2011). It is also understood that the work force cannot be easily reduced, since the servers are, in general, effective and therefore have stability. In addition, “human capital”, the main input of DNPMⁱ is insufficient to meet the demands (TCU, 2011, DNPM, 2017c) and it was further reduced due to the retirement of the civil servants and the lack of public tenders to recompose these vacancies, which increases the need to optimize existing human resources.

It is important to note that the DNPM performance is assessed annually as to the achievement of the organizational objectives, which are set by the entity's Director General (BRASIL, 2004). Regarding the management of mining titles, in 2016, 18,700 requirements were analyzed as detailed in Table 6:

Table 6: DNPM Goals - Management of Mining Bonds - 2016

Mining title	Goal	%
Search permit	16.600	88,8
Licensing	1.700	9,1
Perforation of mining prospector	200	1,1
Extraction log	200	1,1
Total	18.700	100,0

Source: Prepared by the authors based on DNPM (2017c).

The institutional evaluation results have a significant impact on the remuneration of the employees, since 80% of the compensation due to them, which corresponds to approximately 60% of the total remuneration (BRAZIL, 2017. As a consequence of the stipulated goals for the requirements analysis, it is reasonable to assume that DNPM managers have a preference to analyze requirements for a permit, for exploration, for licensing and for permitting mining, and for the latter, indifferent with respect to extraction logs. Therefore, the security regions method was used to incorporate these preferences into classic DEA models:

Table 7: DEA models with preferences used in research

DEA/RCE Model		DEA/RVE Model
	(13)		(14)

Source: Prepared by the authors.

In the implementation of these models, the software Integrated Decision Support System (SIAD), proposed by Meza et al. (2005) was adopted. The results are shown below.

4. ANALYSIS AND RESULTS DEMONSTRATION

In this section, the research results are presented and discussed. Initially, Table 8 compares the efficiency scores of the DNPM superintendencies, calculated through the classic and preferred DEA models.

Table 8: DNPM Supervisors efficiency scores - 2016

N	DNPM Superintendence	Classic DEA			DEA with preferences
		Technical Efficiency		Efficiency to Scale	Technical Efficiency		Efficiency to Scale
		Overall	Pure		Overall	Pure
1	BA	1,00	1,00	1,00	1,00	1,00	1,00
2	MG	1,00	1,00	1,00	1,00	1,00	1,00
3	TO	1,00	1,00	1,00	0,86	1,00	0,86
4	RO/AC	1,00	1,00	1,00	0,68	1,00	0,68
5	PI	1,00	1,00	1,00	1,00	1,00	1,00
6	PR	0,90	0,97	0,93	0,84	0,91	0,91
7	MA	0,65	0,69	0,94	0,58	0,60	0,97
8	GO/DF	0,67	0,82	0,82	0,43	0,46	0,94
9	RS	1,00	1,00	1,00	0,43	0,44	0,98
10	MT	1,00	1,00	1,00	0,40	0,41	0,98
11	SP	0,39	0,40	0,99	0,37	0,37	0,99
12	AL	0,53	0,63	0,84	0,21	0,37	0,55
13	CE	0,53	0,59	0,90	0,32	0,33	0,96
14	SC	0,39	0,39	1,00	0,33	0,33	0,99
15	RJ	0,46	0,53	0,87	0,27	0,28	0,95
16	PB	0,43	0,43	0,99	0,24	0,28	0,84
17	RN	0,45	0,55	0,81	0,27	0,28	0,96
18	MS	0,48	0,49	0,99	0,22	0,27	0,83
19	ES	0,37	0,40	0,94	0,26	0,27	0,97
20	SE	0,37	0,37	1,00	0,25	0,26	0,96
21	PA	1,00	1,00	1,00	0,19	0,19	0,98
22	PE	0,20	0,23	0,88	0,12	0,12	0,96
23	AM	0,12	0,16	0,74	0,09	0,11	0,86
24	RR	0,27	0,31	0,87	0,08	0,08	1,00
25	AP	0,30	0,32	0,93	0,05	0,06	0,96
Mean		0,59	0,65	0,94	0,38	0,46	0,92

Source: Elaborated by the authors according to the research results.

In Table 8, the score analysis calculated by the classical DEA models shows that eight DNPM superintendents (BA, MG, TO, RO / AC, PI, RS, MT and PA) obtained the maximum technical efficiency. According to these models, the average level of overall inefficiency was 41% (1-0.59), which means that the superintendencies evaluated could, on average, increase the number of research permit publications by up to 41% and Licensing grants, mining permits and extraction logs, without increasing the number of servers in the middle and end areas. It is also noted that overall technical inefficiency is due to pure technical (operational) inefficiency, quoted at 35% (1 - 0.65), rather than to inefficiency of scale, whose average was 6% (1 - 0.94).

Still in relation to the results shown in Table 8, it is verified that the incorporation of management preferences reduced the number of efficient superintendencies to five. In addition, it generated technical efficiency scores less than or equal to those calculated by classical DEA models, which, at the 5% level of significance, was confirmed by mean differences tests. These results are due to the fact that the incorporation of management preferences prevented low weights from being attributed to the outputs considered important by the organization.

Table 9 shows that DNPM superintendencies located in BA, MG, TO, RO/AC and PI, presented the highest pure technical efficiency, and therefore are benchmarks for the others. In addition, the current numbers and the targets of publications of research permits and licensing licenses, mining permits and extraction records are highlighted. It should be noted that the targets expose the consequences of inefficiency, since they indicate the outputs that should have been obtained if the superintendencies were efficient. It is noted that if all DNPM superintendencies were efficient, the institutional goals would be fully met, which is not the case in the current situation.

Table 9: DNPM Superintendencies benchmarks and targets - DEA Model with Preferences

N	DNPM Superintendence	Benchmark	Outputs
			perm_pub		lic_granted		perm_granted		reg_granted
			Actual(A)	Target(T)	A	T	A	T	A	T
1	BA	BA	2746	2746	125	125	5	5	6	6
2	MG	MG	2926	2926	203	203	34	34	6	6
3	TO	TO	267	267	46	46	5	5	0	0
4	RO/AC	RO/AC	125	125	33	46	6	6	1	1
5	PI	PI	331	331	36	36	0	0	0	0
6	PR	TO	568	622	47	52	0	25	3	0
7	MA	PI	294	489	34	56	0	1	0	1
8	GO/DF	BA	1057	2305	177	386	2	6	2	6
9	RS	BA	581	1327	251	573	15	36	150	343
10	MT	PI	446	1087	40	97	25	62	7	18
11	SP	BA	708	1897	47	126	0	2	4	12
12	AL	RO/AC	58	156	24	64	0	2	0	5
13	CE	BA	732	1810	76	337	0	11	1	5
14	SC	TO	610	1851	62	188	0	2	15	48
15	RJ	BA	402	1413	84	295	0	3	0	3
16	PB	TO	175	616	21	74	4	17	0	3
17	RN	TO	249	903	62	225	1	6	0	6
18	MS	TO	164	607	44	163	0	3	0	3
19	ES	TO	303	1141	51	192	0	3	1	7
20	SE	PI	95	359	18	68	0	3	0	3
21	PA	BA	438	2314	69	365	45	242	0	4
22	PE	BA	222	1787	45	362	0	7	0	7
23	AM	PI	75	689	12	110	0	8	0	8
24	RR	PI	15	185	13	160	0	11	0	11
25	AP	PI	28	497	7	124	4	88	0	17
Total			13615	28450	1627	4473	146	588	196	523
Goal			16600		1700		200		200

Source: Elaborated by the authors from the research results.

It is important to note that although the DEA makes it possible to calculate the efficiency scores of the DMUs, this technique alone does not identify the factors that affect them (ZHU, 2015). In order to complete the analysis, we verified the effect of the number of protocols for search permits (alv_prot), licensing (lic_prot), mining permits (perm_prot) and extraction logs (peg_prot) in the superintendencies of the DNPM (the descriptive statistics of these variables in the appendix), which cannot be controlled by the managers, on the pure technical efficiency scores calculated by the DEA model with preferences. The option to consider pure technical efficiency scores as a dependent variable is justified because, as already pointed out, in the case of DNPM superintendencies, the overall technical inefficiency is due more to pure technical inefficiency than to inefficiency of scale.

In the estimation of the model coefficients, given by (15), the tobit regression was adopted, the most indicated when the dependent variable is censored (WOOLDRIGE, 2006), as is the case with efficiency scores, which are unit limited.

(15)

The estimates results are shown in Table 10. It is observed that the overall regression significance is guaranteed, according to the statistic X². Other tests showed that there were no problems of collinearity, endogeneity, heteroscedasticity or abnormality of the residues.

Table 10: Effects on the pure technical efficiency scores calculated by the DEA model with preferences

Variable	Coefficient	Standard-Error	z	P-value
Constant	0,2223	0,0745	2,9860	0,0028
alv_prot	0,0003	0,0001	2,2920	0,0219
lic_prot	0,0010	0,0013	0,7879	0,4308
perm_prot	-0,0037	0,0003	-1,4530	0,1463
peg_prot	-0,0037	0,0298	-1,2490	0,2116
X² statistics			25,89	0,000

Source: Elaboration of the authors from the research results.

After analyzing the econometric indicators, we discuss the significance and the sign of the estimated coefficients highlighted in Table 10. Thus, at the 5% level of significance, only the independent variable alv_prot had a significant effect on the scores of pure technical efficiency of DNPM superintendencies. The positive sign of the variable coefficient indicates that the more research requirements are filed, the greater the efficiency of superintendencies tends to be. This is an indication that the superintendencies in which more research requirements are filed may be under pressure to analyze them more quickly, which is consistent with the fact that the institutional goals of analysis of those processesare preferred by managers.

5. FINAL CONSIDERATIONS

In this paper, we utilized DEA models to evaluate the efficiency of DNPM's superintendence in 2016. For this purpose the servers in the end-and-a-half areas and the number of published and granted mining titles were the inputs. This paper deals with the managers preferences by means of weight restrictions, using the safety regions method.

The DEA models scores revealed that, from the 25 DNPM superintendents, eight (BA, MG, TO, RO / AC, PI, RS, MT and PA) obtained maximum technical efficiency (global and pure). By incorporating the preferences of managers in relation to the outputs, it was verified that only three superintendence (BA, MG and PI) were globally efficient and that, like these, the superintendence of TO and RO/AC also had maximum efficiency pure technique.

It was also found that in both models, the overall technical inefficiency of DNPM superintendence is due more to the pure technical inefficiency than to the inefficiency to scale. In this sense, considering the model that incorporates managers' preferences, it was verified that the number of mining titles published and granted could increase by 62% without changing the workforce. It was also found that DNPM superintendence tend to be more efficient. This is not surprise, since the institutional goals contributes to manager's preference. After all, the compensation of DNPM's employees is directly related to the goals fulfillment.

This work has highlighted importance on planning at the efficiency level, demonstrating benchmarks and realistic performance targets to the DNPM's superintendence. The present findings might help to solve the pending processes analysis by the autarky, thus promoting investments to the mineral sector, which is of the utmost importance for the country. We hope that our research will serve as a basis for future studies on the governmental efficiency.

REFERENCES

ALCÂNTRA, A. A. M.; SANT’ANA; A. P.; LINS, M. P. E. (2003). Restringindo flexibilidade de pesos em DEA utilizando regressão MSEA. Pesquisa Operacional, v. 23, n. 2, p. 347-357.

ALLEN, R.; ATHANASSOPOULOS, A.; DYSON, R. G.; THANASSOULIS; E. (1997) Weights in restrictions and value judgements in Data Envelpment Analysis: evolution, development and future directions. Anals os Operations Research, n. 73, p. 13-34.

ARAÚJO, R. M.; DIAS, T. F.; GOMES, J. F. (2015). Eficiência Processual no Judiciário: aplicação do DEA em varas únicas no Rio Grande do Norte. Nucleus, v. 12, n. 2, p. 111-134.

BANKER, R. D.; CHARNES, A.; COOPER, W. W. (1984). Some models for estimating technical and scale inefficiencies in data envelopment analysis. Management Science, n. 30, v. 9, p. 1078-1092.

BRASIL (2004). Lei n.º 11.046, de 27 de dezembro de 2004. Dispõe sobre a criação de Carreiras e do Plano Especial de Cargos do Departamento Nacional de Produção Mineral - DNPM e dá outras providências. Diário Oficial da União, Brasília.

BRASIL (2017). Tabela de Remuneração dos Servidores Públicos Federais Civis e dos Ex-territórios. Ministério do Planejamento, Desenvolvimento e Gestão, Secretaria de Gestão de Pessoas e Relações de Trabalho no Serviço Público, v. 69, jan.

CHARNES, A.; COOPER, W. W.; RHODES, E. (1978). Measuring the efficiency of decision making units. European Journal of Operational Research, v. 2, n. 6, p. 429-444.

CHARNES, A.; COOPER, W.; WEI; Q. L.; HUANG; Z. M. (1989). Cone Ratio Data Envelopment Analyisis and multiple objective linear programming. International Journal of Management Science, n. 20, v. 7, p. 1099-1118.

COOK, W. D.; TONE, K.; ZHU, J. (2014). Data envelopment analysis: Prior to choosing a model. Omega, v. 44, p. 1-4.

COOPER, W. W.; SEIFORD, L. M.; ZHU, J. (2011) Handbook on Data Envelopment Analysis. Springer Science, New York.

DINIZ, J. A.; LIMA, H. M. V. (2015). Eficiencia na aplicação dos recursos públicos pelos tribunais de justiça do Brasil. InterScientia, v. 2, n. 3, p.78-98.

DNPM – Departamento Nacional da Produção Mineral. (2017a). Estatísticas. Disponível em: < http://www.dnpm.gov.br/acesso-a-informacao/estatisticas>. Acesso em: jun. 2017.

DNPM – Departamento Nacional da Produção Mineral. (2017b). Portal da Outorga: regimes. Disponível em: < http://outorga.dnpm.gov.br/SitePages/regimes.aspx>. Acesso em: jun. 2017.

DNPM – Departamento Nacional da Produção Mineral. (2017c). Relatório de gestão – exercício 2016. Disponível em: < http://www.dnpm.gov.br/acesso-a-informacao/prestacao-de-contas-1/relatorio-de-gestao-exercicio-2016/view>. Acesso em: jun. 2017.

DNPM – Departamento Nacional da Produção Mineral. (2017d). Serviço de Informação ao Cidadão – SIC. Disponível em: < http://www.dnpm.gov.br/acesso-a-informacao/servico-de-informacao-ao-cidadao-sic >. Acesso em: jun. 2017.

EMROUZNEJAD, A.; YANG, G. L. (2018) A survey and analysis of the first 40 years of scholarly literature in DEA: 1978–2016. Socio-Economic Planning Sciences, v. 61, p. 4-8.

FERREIRA, C. M. D. C.; GOMES, A. P. (2009). Introdução à análise envoltória de dados: teoria, modelos e aplicações. UFV.

GONÇALVES, A. C. (2010). Definição das restrições aos pesos em Análise Envoltória de Dados (AED) por correlação canônica e regressão linear. 2010. Tese (Doutorado). Instituto Alberto Luiz Coimbra de pós-graduação e pesquisa em engenharia, Universidade Federal do Rio de Janeiro, Rio de Janeiro.

IBRAM – Instituto Brasileiro de Mineração. (2015). Informações sobre a economia mineral. Disponível em: < http://www.ibram.org.br/>. Acesso em: jun. 2017.

INEP – Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2017). Ideb. Disponível em: < http://portal.inep.gov.br/ideb>. Acesso em: jun. 2017.

JORO, T.; KORHONEN, P. J. (2015). Extension of Data Envelopment Analysis with preference information. International Series in Operations Research & Management Science.

LI, Y.; LEI; X.; DAI, Q.; LIANG, L. (2015). Performance evaluation of participating nations at the 2012 London Summer Olympics by a two-stage data envelopment analysis. European Journal of Operational Research, v. 243, n. 3, p. 964-973.

LIU, J. S.; LU, L. Y., LU, W. M.; LIN, B. J. (2013). A survey of DEA applications. Omega, v. 41, n. 5, p. 893-902.

LIU, W.; SHARP, J.; WU, Z. (2006). Preference, production and performance in data envelopment analysis. Annals of Operations Research, v. 145, n. 1, p. 105-127.

MACEDO, M. A. S.; NOVA, S. P. C. C.; ALMEIDA, K. (2010). Mapeamento e análise bibliométrica da utilização da Análise Envoltória de Dados (DEA) em estudos em contabilidade e administração. Contabilidade, Gestão e Governança, v. 12, n. 3, p. 87-101.

MEZA, L. A.; BIONDI NETO, L.; MELLO, J. C. C. B. S.; GOMES, E. G. ISYDS. (2005). Integrated System for Decision Support (SIAD - Sistema Integrado de Apoio à Decisão): a software package for data envelopment analysis model. Pesquisa Operacional, v. 25, n. 3, p. 493-503.

MME – Ministério de Minas e Energia. (2011). Portaria nº 247, de 8 de abril de 2011. Aprova o Regimento Interno do Departamento Nacional de Produção Mineral. Brasília-DF.

MOTTA, P. R. M. (2013). O estado da arte da gestão pública. Revista de Administração de Empresas, v. 53, n. 1, p. 82-90.

NOGUEIRA, J. M. M.; OLIVEIRA, K. M. M.; VASCONCELOS, A. P.; OLIVEIRA, L. G. L. (2012). Estudo exploratório da eficiência dos Tribunais de Justiça estaduais brasileiros usando a Análise Envoltória de Dados (DEA). Revista de Administração Pública, v. 46, n. 5, p. 1317-1340.

OLIVEIRA, L. G. L.; NOGUEIRA, J. M. M.; OLIVEIRA, K. M. M.; OLIVEIRA FILHO, S. M. (2016). Medição da eficiência de magistrados e de unidades judiciárias no Ceará, Brasil: o sistema Eficiência. jus. Cadernos EBAPE. BR, v. 14, n. 3, p. 836-857.

ROLF, F.; SHAWNA, G.; DIMITRIS, M. (2015). Advances In Data Envelopment Analysis. World Scientific.

ROLL, Y; COOK, W. D.; GOLANY; B. (1991). Controlling factor weights in DEA. IIE Transactions, v. 23, n.1, p. 2-9.

SARTORI, S. (2016). Proposta de método de avaliação integrada de sustentabilidade com uso da análise envoltória de dados. Tese (doutorado). Programa de Pós-Graduação em Engenharia de Produção da Universidade Federal de Santa Catarina.

SHIROUYEHZAD, H.; YAZDANI, F. (2014). Performance evaluation and ranking of participation Asian countries in 2012 London Olympic Games through Data Envelopment analysis. Journal of Data Envelopment Analysis and Decision Science, v. 2014, p. 1-11.

SILVA, J. S.; CORRÊA, C. R.; GOMES, A. P. (2017). Determinantes da eficiência dos programas de pós-graduação em Economia do Brasil. Reflexões Econômicas, v. 2, n. 2, p. 55-75.

TCU (2011). Tribunal de Contas da União. Relatório de auditoria TC 011.720/2011-5. Disponível em: < www.tcu.gov.br/consultas/juris/docs/judoc/acord/20111125/ac_3072_51_11_p.doc>. Acesso em: jun. 2017.

THANASSOULIS, E.; PORTELA, M. C.; ALLEN, R. (2004). Incorporating value judgments in DEA. In: Handbook on data envelopment analysis. Springer US, p. 99-138.

THOMPSON, R. G. et al. (1990). The role of multiplier bounds in efficiency analysis with application to Kansas farming. Journal of Econometrics, v. 46, n. 1-2, p. 93-108.

TOMAZ, R. (2015). Morosidade do DNPM na liberação de licenças de pesquisas reduz o aporte. Diário do Comércio, 15 jan. 2015. Disponível em: < http://www.diariodocomercio.com.br/noticia.php?tit=morosidade_do_dnpm_na_liberacao_das_licencas_de_pesquisas_reduz_aportes&id=128724>. Acesso em: jun. 2017.

WONG, Y. H. B.; BEASLEY, J. E. (1990). Restricting weight flexibitity in DEA. Journal of the Operation Research Society, v. 41, p. 829-835.

WOOLDRIDGE, J. (2016). Introductory Econometrics: A Modern Approach, 5 ed. Pioneira Thomson Learning.

ZHU, J. (2014). Quantitative models for performance evaluation and benchmarking, 3 ed. Boston: Springer.

ZHU, J. (2015). Data Envelopment Analysis: a handbook of models and methods, 1 ed. Boston: Springer.

APPENDIX

Table 11: Descriptive statistics of the requirements filed in the superintendence’s of DNPM - 2016

Variables	Minimum	Maximum	Mean	Stand. Deviation
perm_pub	42	2761	560,40	655,23
lic_granted	8	392	108,72	100,49
perm_granted	0	615	62,16	139,18
reg_granted	0	119	8,52	23,69

Source: Prepared by the authors based on data from DNPM (2017a)

ⁱ In 2016, the expenses committed by DNPM were R $ 311,189,292.24. Of this amount, R $ 253,006,130.64, equivalent to 81.3% of the total, were allocated to the payment of salaries and social charges of the serve (DNPM, 2017c).