PERFORMANCE INDICATORS: A COMPARATIVE ANALYSIS
BETWEEN PUBLIC AND PRIVATE COLLEGES IN BRAZIL
Átila de Melo Lira
Universidade Paulista, Brazil
E-mail: atilalira@hotmail.com
Irenilza de Alencar Naas
Universidade Paulista, Brazil
E-mail: irenilza@gmail.com
Submission: 21/08/2014
Revision: 08/09/2014
Accept: 21/10/2014
ABSTRACT
A comparative analysis between the use of performance
indicators to public and private organizations have always been required to
examine the scenario related to both. This study seeks to analyze the use of
Balanced Scorecard (BSC) to identify and understand the main differences and
similarities in public and private higher education institutions (HEIs) in
Brazil in relation to the use of other organizations performance indicators. A
quantitative and exploratory approach was adopted using institutional documents
analysis. Data was searched on the websites of Brazilian higher education
public and private organizations in order to accomplish this analysis
comparative. The results showed that even reviewing few public institutions the
use of performance indicators appears to be more efficient than those applied
to the private ones. Private universities should observe and improve their
processes and performance indicators based on those used in Brazilian public
universities. This initial research still opens a horizon so that other studies
be developed within this thought stream.
Keywords: high
educational institutions, public education, Balance scorecard.
1. INTRODUCTION
There
is a theoretical gap in relation to studies on the use of this measurement tool
in the context of public educational organizations, especially in relation to
the use of performance indicators.
Much
of the research focusing on the application of the BSC in higher education
institutions (HEIs) ignores the public sphere covering only the private institutions,
then, this research shows its importance to broaden the discussion to new
horizons of inquiry, not only addressing the BSC applied in public HEIs, given
the aforementioned theoretical gap, but performing a systematic and linear
comparison between these and the BSC adopted by private higher education
institutions, in order identify the main similarities and differences between
these two specific approaches.
This
study aims to give greater significance to a topic still under construction,
since much of the research dealing with the use of the BSC in higher education
institutions (HEIs) to ignore the public sphere covering only private in
nature.
2. LITERATURE REVIEW
2.1. Adaptation
of the BSC nonprofit educational institutions
The
methodology for measuring and assessing organizational performance known as
Balanced Scorecard (BSC) has been widely used in the management of private HEIs
system. However, initiatives aiming at the use of this tool within the public
scope are still underdeveloped. Despite its focus on results, there are a few
experiences where public higher education institutions use performance
indicators to measure and monitor their activities in a systematic way
(OLIVEIRA, 2001).
This
factor is related to the initial design itself given to the BSC, which
according to Steele (2001) is to meet the demand of private companies through
strategic mechanisms for measuring and evaluating the activities and results. However,
Kaplan e Norton (1997) argues that the propositions of the BSC should serve as
a model and not as a standing instrument evidencing their adaptability to other
types of organizations.
The
adaptation of the BSC to other contexts is not simple. In current literature,
authors Kaplan e Norton (1992) admits that the use of this tool in nonprofit
organizations is different and to make it suitable in this context changes, and
adjustments must be performed (KAPLAN, 1994).
One
of these changes relates to the replacement of a conditional business
perspective view. Moreover, it has been observed that public HEIs operate in a
more ambiguous, complex and unique environment (CHETKOVICH; FRUMKIN, 2003;
KONG, 2007) and model BSC fails to account for this complexity (KONG, 2009). It
is noticed that different from private institutions, the non-profit institutions
primarily engaged in producing maximum results using the minimum possible
resources, directs yours activities to achieve the organizational mission and
not to obtain financial income (KAPLAN; NORTON, 1997).
2.2. The
issues involved in measures of organizational performance
Most
failures of the BSC approach are associated with poor management of performance
metrics. According to the Hackett Group's researchers, less than 20% of
companies those have invested in BSC implementations that are creating value. A
common mistake that is discernible from the study of the BSC in universities is
that they tend to rely on many metrics (NEELY, 2003). Companies often use
metrics overused. In current literature Neely (2003), it is found that the
current administration is in crisis because it is "drowning in data"
Bourne (2002) adds that the design of performance measurement systems focuses
on how many engines will use instead of focusing on the quality of these
mechanisms.
Interestingly,
unlike for-profit companies, the performance measures of public universities
reveal very little about their internal business data. Furthermore, the number
of steps does not present a balanced approach. Studies show that just over 10%
of the measures relate to the business perspective, being most prevalent
measures for the stakeholder view (BOURNE, 2002).
Some
universities have implemented the BSC in individual units, resulting in
performance measures, which take into account only the performance of the unit.
Although it seems logical, it can result in sub - optimization. Not relate to
the key performance indicators with metrics often universities tend to measure
only a small part of what really matters (KONG, 2009).
2.3. Methodology
This
study was guided by the following question: What are the main differences and
similarities between the approach of the BSC in public and private institutions
of higher education regarding the use of performance indicators? We believe
that this study might identify and understand the main differences and
similarities between the approach of the BSC in public and private HEIs in
relation to the use of performance indicators.
In
order to enable a better understanding of the facts to be presented, this work
will be organized in four sections, in addition to this it is, evident. It
begins with a critical review of the literature around the BSC and its
adaptation public HEIs. Subsequently the methodological procedures are pointed
in sequence are reported the results and discussions. Finally, the concluding
remarks of the study are presented.
Implementation
strategies of this study departed from a deductive research process, where the
conclusion of the investigation is evidenced implicitly in their premises. This
method is developed in such way that a certain general analysis is able to
generate particular inference on an actually observed occurrence (MARCONI;
LAKATOS, 2004). However, in order to allow a more accurate analysis of the
phenomenon, also reported to a qualitative approach, as a complement to the
central strategy of the research process.
This
is an exploratory research, aiming to familiarize the researcher with a subject
still little explored, without theoretical or practical implications, which allow
the construction of evidence in the empirical reality (AZEVEDO, 1999).
As
for the methods for the theoretical development of the study document analysis
was used, as this is an important technique not only to complement the
information obtained from other sources, but also for its ability to reveal new
findings of a problem (AZEVEDO, 1999). This study was used as a source of
information the web sites of Brazilian public and private universities to raise
research useful documents. This technique differs from longitudinal studies
where data are collected over a long period (RODRIGUES, 2012).
Interviews
with members of public and private institutions studied were also performed.
Using a script semi structured interviews, managers of 10 public institutions
and 06 private institutions of scope were asked about the results of the
documentary analysis undertaken. The interviews were conducted through online
form with the actors. It is noteworthy that the small number of involved are
the result of the fact that no other universities have taken part in the research.
To
meet the objectives originally proposed, this study considered the totality of
existing federal public and private universities in Brazil in order to identify
and understand their similarities and differences in the use of performance
indicators. Data were collected through a search on the web sites of all
Brazilian public and private universities; this search has focused on the
search for performance indicators.
Thus,
were analyzed the indicators related to research and extension education
activities, learning, institutional management, financial management and
communication services. Furthermore, were still mentioned in the analysis, indexes are not as common in the context of
institutions whether public or private.
The
results obtained through the confrontation of information pertaining to federal
public universities, and private universities were interpreted by techniques
specific to the objectives proposed analysis. The results that emerged from the
research were placed in a straight sequence of significance to provide
explanations that allow making inferences within a reality analyzed (RODRIGUES,
2012).
2.4. Results
and Discussion
In
this section, the results of the study will be presented and discussed their
theoretical and practical implications. It was attempted by means of analytical
techniques and particularly compatible with the proposed objectives provide
useful information for making clear differences between the use of performance
indicators in public and private HEIs.
2.5. Comparative
regarding the use of performance indicators
When
comparing the BSC approach adopted by the Brazilian public and private HEIs the
first notable consideration is that both apply performance indicators to
measure and analyze related activities, namely research and extension, teaching
- learning, institutional management, business management, and communication
services. Importantly, as the business management is a small difference, that
while public HEIs under administer the funds from the government of a private
nature deal with financial income derived from third parties who benefit from
their services.
There
are large differences in the use of performance indicators between institutions
of two areas that deserve to be highlighted. Figure 1 shows some peculiarities
which enable this differentiation between the BSC approaches in these two
specific types of the educational institution.
Figure 1: Comparison between Brazilian HEIs regarding the
application of performance indicators (ID).
Some
differences are evident, starting with the amount of existing private institutions
in the country, totaling 91 institutions, much higher number public HEIs
totaling only 60 units (Figure 1). Even with this difference, public universities
outnumber private in the adoption and use of performance indicators. Forty
three private HEIs have this measurement and analysis tool for assessing the
results of its activities for 45 public universities. What seems like a minor
difference becomes large when considering the context in which it appears, therefore,
47 % of HEIs private origin has performance metrics while in public under the
adoption of this tool reaches 75 % of the organizations.
It
should be stressed that this preponderance of public HEIs as the adoption of
indicators may be related to the higher number of stakeholders associated with
these organizations. This term refers to stakeholders, including students,
teaching and non- teaching, government and their development agencies, accreditors,
auditors and advisors, and even society (BURROWS; HARVEY, 1992). The CFO of a
public university analyzed highlights this feature in his speech:
“[…] we have to be accountable not only to the
directors, as occurs in most private institutions, our responsibility is much
greater, because the number involved in the performance and results of our
activities is considerably larger, government, funding agencies, auditors ...
Anyway, there are many conflicting interests to administer” (Public University 08).
Regarding
IES using performance indicators – one realizes that there is variation in the
degree of measurement activities. The activity of research and extension data
are clear in showing the dominance of public HEIs in the metrics within the
analysis that affect the aforementioned functions. While 26 public institutions
measure and analyze this activity, only 11 has a private interest. Credited
with this observation the greatest incentive given to research and extension in
Brazilian public universities which consequently implies the existence of mechanisms
capable of measuring this action (INEP, 2012).
“Here we have a
major concern with the knowledge that we are producing, students are encouraged
to develop research and provide all the conditions for this, laboratories,
libraries with large collections. [...] The same way, we register everything
that is produced, to have indicators that allow us to analyze the development
of our research activities”. (Public University 01).
As to
teaching – learning one realizes that, these seemingly private institutions are
preferable. This criterion was the only one where there was a preponderance of
that instance on the public sphere. This finding demonstrates that private HEIs
are more interested in measuring and evaluating the performance of their
teachers in the implementation of their activities and their satisfaction with
students to educational services offered, than the public ones.
In
turn, institutional management is the activity where there was a major discrepancy
between the data obtained. For while 38 public HEIs adopt relevant metrics for
this activity only 10 private institutions move actions in order to measure the
results of their activities and institutional management. Public universities
are more concerned to know, using metrics, effectiveness and coordination of
sectors, departments, library and laboratories, as indicated by the data.
"[...] Public institutions demonstrate a greater
interest in integrating activities related to institutional management, because
they know the benefits from this integration ranging from the improvement of
services offered to students up to a substantial cost reduction" (Public
University 06) .
Financial
management was another activity where the results indicated a prevalence of
public institutions as under the use of performance indices. Although not for
profit organizations devoted to Brazilian public universities bother to
demonstrate to stakeholders how the resources are being used to get through
indicators for this purpose. In the information services, related to all the
analyzed context public HEIs have a greater tendency to use metrics in order to
measure the effectiveness of such services. It is significant that the rate does
not claim to measure the quality of services since the results would be inaccurate,
due to the subjective nature of this aspect, which is actually measured, is the
effectiveness of services.
It is
worth mentioning the presence of specific indexes in some higher education
institutions in the country (were not mentioned in Figure 1), as the ratio of
aid to students that, in general, refers to benefits such as food assistance,
housing assistance, scholarship permanence, emergency aid and worth book
(allows students to purchase books for cheaper prices). Although only six
public institutions and derisory, three private universities have records and
indicators for the evaluation of these services their mention is needed in this
analysis, since one must consider the importance and the social impacts of
these indicators (approximately 5 million students benefit from these aids,
only in public institutions) and also its implications on financial resources
of these institutions.
Another
index that shows restricted to a few institutions of higher education, whether
public or private, refers to artistic activities and sports and leisure. Only
two Brazilian universities, these public sphere, have in their records
indicators for the analysis of these activities. According to the directors
responsible for the department of physical education course, of one such
institution that adopts indicators related to these activities:
"Our business is likely to remain restricted to
our department, there is no effort to make them something public, an indicator
that can be compared with other institutions. This same difficulty is found in
other institutions" (Public University 04).
The
environmental impact of the activities carried out by the institutions was
another overlooked factor when analyzing the presence or absence of performance
indicators with respect to the garbage collection or effort in reducing the use
of paper activities through recycling efforts or yet the adoption of clean energy
(renewable). Only three Brazilian universities, with two private, have data
controls, performance indices over his actions regarding environmental responsibility.
This is worrying, especially within the context we live where concern for the
environment is increasingly present.
2.6. Different
approaches to the financial perspective
Public
and private higher education institutions in Brazil have different characteristics
when analyzing your financial outlook. The latter, as its very nature, are
profit-oriented, ie, improve their activities towards achieving satisfactory
financial result, while public HEIs develop its activities to meet the
organization’s mission (BARMAN, 2002).
Kaplan
e Norton (1992) argue that, within the private institutions, the structure of
the BSC has the financial perspective as their main goal. Adapting this model
proposed private educational institutions are perceived that the business
perspective, goes much further, as it represents not only its outcome, but also
the starting point of the flow of activities pertaining to these organizations.
In turn, the public HEIs has a different structure. The model below sets out
the main difference between these two perspectives.
Figure 2: Model of comparative analysis of Brazilian
private and public educational institutions.
As
for private organizations, this model suggests, (1) that these companies need
financial resources to initiate their activities and (2) all activities are gathered
toward obtaining financial returns, ie
private HEIs run profit towards the profit himself. This finding justifies the
idea of flow given the economic outlook adopted by Brazilian private HEIs.
"In reality, provide the knowledge our students
is our reason for being, However we cannot close our eyes to the profit for the
financial resources, we depend on them. Regardless of all we are a private company,
the profit moves us [...] "(University Private 02).
Different
with public HEIs (Figure 2) as despite having as a starting point" sufficient
financial income" their activities do not make profits; they are oriented
to achieve the organizational mission. Like all organizations, or nonprofit,
public HEIs need financial resources for the development of their activities,
however, their main goal is to use these resources efficiently, ie run maximized shape with minimal of
resources possible.
3. EFFECTIVENESS OF THE RESULTS OF THE PERFORMANCE
INDICATORS
Performance
indicators, or metrics, are used in the context of HEIs to measure and evaluate
its activities and results; however, one realizes that the adoption of these
indices has an ambiguous behavior as to guarantee a good performance of these
institutions. Forty three private HEIs use performance metrics; however, when
looking at the result of research undertaken by the Ministry of Education
(MEC), which highlighted the 50 best Brazilian higher education institutions in
the year 2012, only 3 are private. Given this data it is perceived that there
is no clear link between the use of performance indicators and the development
of HEIs. Importantly, this data relates only institutions of a private nature
(INEP, 2012).
By
analyzing the context of public HEIs, the reality is different. It appears that
these organizations the performance indices are strong indicators of the
quality and effectiveness of the services of these organizations. This factor
can be evidenced by the list that shows the best of Brazilian higher education
institutions where among these there is a high prevalence of public
universities (INEP, 2012).
4. FINAL REMARKS
In
Brazil, there is a greater number of private HEIs compared the educational
institutions of the public sphere. While these first totaling 91 institutions,
public universities represent only 60 units. However, there is a greater
tendency to use performance indicators by public HEIs. This finding may be
associated with the complexity of the interests of stakeholders requiring these
companies effective measurement mechanisms that can accurately demonstrate the
expected results.
When
analyzing the HEIs using performance indicators, it was noticed that in respect
of research and extension, communication, institutional management and
financial management services activities there was a preponderance of public
organizations in adopting them. Regarding the activity of teaching - learning,
private HEIs were more favorable, this, in turn, was the only one of all activities
where private institutions predominated up on public.
Another
clear finding in this investigation is the different financial perspectives
associated with public and private institutions. While the public nature of
HEIs has the financial resources your starting point because they need these resources
to develop their activities, private institutions go much further, taking the
financial resources not only as the kick, but as the outcome. HEIs just run
profit towards the profit.
Through
the analysis of the results of research conducted by the Ministry of Education
(MEC), which highlighted the top 50 higher education institutions in Brazil in
2012, it was shown that the adoption of performance indicators in private HEIs
has no direct relationship with the quality of the activities and the
consequent results of these organizations. In contrast, public institutions
adopting performance indices are strong indicators of the quality and effectiveness
of services provided. This evidence demonstrates the ambiguous behavior regarding
the use of performance indicators in HEIs of the two instances.
Finally,
it is expected that the results that emerged from this research can be used as
a reference for building studies within the theme to be here. The field of
study that deals with the application of performance indicators HEIs lies eager
investigations that enable greater contributions with respect to comparative
analysis between public and private HEIs.
REFERENCES
AZEVEDO, I. B. (1999) O Prazer da Produção Científica. São
Paulo: Unimep.
BARMAN, E. A. (2002) Asserting difference: the
strategic response of nonprofit organizations to competition. Social
Forces, v. 80, p. 1191-1222.
BOURNE, M. (2002) The Emperor’s new scorecard. Financial World, p. 48-50.
BURROWS, A.; HARVEY, L. (1992) Defining quality
in higher education: the stakeholder approach, paper to the AETT. In: CONFERENCE
ON QUALITY IN EDUCATION,1, New York, Proceedings … New York: University of York.
CHETKOVICH, C.; FRUMKIN, P. (2003) Balancing
margin and mission nonprofit competition in charitable versus fee-based
programs. Administration
and Society, v. 35, p. 564-596.
INEP (2012) Instituto Nacionais de
Estudos e Pesquisas, Available: http://www.publicacoes.inep.gov.br/resultados.asp?cat=6&subcat=5.
Acessed in: february 23. 2014.
KAPLAN, R. S.; NORTON, D. P. (1992) The
balanced scorecard: measures that drive perfomance. Harvard Business Review. v.70, p. 71-79.
KAPLAN, R. S.; NORTON, D. P. (1997) The balanced scorecard: translating strategy
into action. Boston: Harvard Business school press.
KAPLAN, R. S. (1994) Devising a balanced
scorecard matched to business strategy”. Strategy
& Leadership, v. 22, p. 15-25.
KONG, E. (2009) Facilitating learning through
intellectual capital in social service nonprofit organizations. International Journal of Learning,
v. 16, p. 531-550.
KONG, E. (2007) The strategic importance of
intellectual capital in the nonprofit sector. Journal of Intellectual Capital, v. 8, p. 721-731.
MARCONI, N. A.; LAKATOS, E. M.
(2004) Metodologia Científica. São
Paulo: Atlas.
NEELY, A. (2003) Gazing into the crystal ball:
the future of performance measurement. Perspectives on Performance, v. 2, p.
12-13.
OLIVEIRA, J. (2001) The BSC: an integrative
approach to performance evaluation. Healthcare
Financial Management, v. 55, p. 42-46.
RODRIGUES, T. K. A. (2012) Caracterização de MPEs brasileiras quanto
ao conhecimento de ferramentas de gestão. Dissertação (Mestrado em Administração).
Lavras: Available in: december/ Universidade Federal de Lavras. Available: http://repositorio.ufla.br/handle/1/440.
Acessed in: january 11, 2014.
STEELE, J. (2001) Transforming the balanced
scorecard into your strategy execution system. Manage, v. 1, p. 22-23.