ENGLISH PROFICIENCY TEST IN A LOGISTICS POSTGRADUATE COURSE ADMISSION

Ana Beatriz Aquino

Instituto Federal São Paulo, Brazil

E-mail: anabeatriz.aquino02@gmail.com

Eliana Kobayashi

Instituto Federal São Paulo, Brazil

E-mail: likobayashi@ifsp.edu.br

Submission: 2/24/2020

Revision: 2/29/2020

Accept:3/2/2020

ABSTRACT

The construction of English proficiency tests applied in college admissions is a complex activity as their content must reflect the language skills required in the program and their results can affect applicants’ lives. This study aims to establish the specifications of English language skills to be tested in an admission test for a postgraduate course in Logistics and Operation of a public institution. The qualitative methodology of this investigation encompasses the course documents analysis, the results of past tests, as well as the application of a questionnaire to the professors and the program coordinator. The investigation has a multidisciplinary approach, integrating different areas like Applied Linguistics and Administration. The preliminary results show that English language is considered important in the program documents although it is not clearly stated. In addition, professors seem to have different expectations on student English level but agree on the main linguistic skill.

Keywords: English Proficiency Tests; Admission test; Logistics Postgraduate Course

1. INTRODUCTION

The hegemony of English language in the fields of science, technology and business is already established. This scenario shows the relevancy of English skills to build scientific knowledge and critical thinking, since the linguistics abilities are key to access information and develop an active posture in a globalized world.

Therefore, Brazilian educational institutions play an important role in the construction and development of students’ language skills. Furthermore, one of the goals of the educational institutions is the internationalization as established by CAPES (BRASIL, 2017). However, English teaching in public institutions has presented several problems such as low proficiency level of the students and the lack of frameworks of references and specified competences to be developed by the undergraduate students (KOBAYASHI; GALLARDO, 2019).

In Brazil, the guidelines that regulate the abilities students must have in English, especially in postgraduate courses are not presented clearly. Nevertheless, in order to test students’ proficiency level in admission process, the expected competencies must be clearly stated so that applicants can understand how to succeed in the test.

This kind of English test is classified as proficiency tests since they aim to assess whether students' language skills meet the requirements of the course and whether students will be able to carry out activities involving English during the classes. Thus, it’s necessary to understand the specific skills to be developed during the course to build a test that meets such needs.

Therefore, this investigation focusses on the competences that must be addressed in an English test applied in a Logistics and Operation postgraduate course admission offered by a public institution located in the Great São Paulo - Brazil.

2. LITERATURE REVIEW

The hegemony of English, present in 85% of international companies as one of the most widely used languages, and the main language in the areas of science and technology (GRADDOL, 2000), contributes to the argument that educational institutions should focus on the development of their students' language skills and also to internationalize Brazilian high education institutions (BRASIL, 2017).

The development of a critical citizen in a globalized era involves the ability to use the language which allows access to information, enables acting in society and contributes to the development of a critical view (LANKSHEAR, 1997). In addition, in the work market companies usually apply English tests to analyze the proficiency level of the applicants (KOBAYASHI, 2010). On the other hand, English teaching in public institutions has presented problems and revealed the low proficiency of Brazilian students (LEFFA, 2005; PAIVA, 2011).

Admission tests can be classified as proficiency tests which means they analyze how applicants can use the language in future situations regardless of their learning experience (HUGHES, 1989). As Vallete states (1977, p.6), "the purpose of the proficiency test is to determine whether the language skills (of the students) match the specific requirements". In the case of postgraduate courses, the test aims to assess whether the student is competent to carry out the activities involving the language during classes.

The Logistics area has grown considerably recently so much that technology courses have become viable alternatives to provide faster employability to their graduates. According to data from the 2016 Higher Education Census (BRASIL, 2018), these degrees represented 19.86% of the total higher education courses offered. Specifically in ‘Management and Business area, the Technology in Logistics is placed second in number of vacancies in undergraduate courses ranking, representing a total of more than 22% of the area courses, including bachelor's degrees. This index has a direct impact on the creation of postgraduate courses in Logistics with many divisions such as Logistics and Operations, Logistics and Supply Chain, Logistics Corporate, Logistics Management, and others, considering only a few public and private institutions located in Greater São Paulo.

Despite the relevance of the English language in society emphasized by globalization process, Resolution No. 1 of April 6, 2018, from the Ministry of Education, which regulates lato sensu postgraduate courses in Brazil, presents only general guidelines for the program operation, as well as general components of the Pedagogical Course Project (PPC) that is restricted to workload, obligation of course plans, faculty composition and student assessment. Thus, the requirements concerning English knowledge to enter the program are under the responsibility of the educational institutions. On the other hand, the English requirements set by the work market and the worldwide importance of the language in the global society justify its relevance.

It is expected that a postgraduate course in Logistics will require intensive English language use to read international papers, as well as cases published in international media, participation in academic events among others. This expectation cannot be restricted to the student's performance in areas that integrate Logistics and Foreign Trade or Logistics and Supply Chain, but considering the general training of a postgraduate student.

Systematic searches carried out in national and international databases with key terms combined as logistics, English and competences have not resulted in studies that have joined efforts to raise the language skills needed by professionals working in Logistics field of logistics. A survey with the combination of these words in the national database SPELL (Scientific Periodicals Electronic Library) did not result in studies that addressed the combined themes. The search made in the Web of Science database resulted in 24 articles, none of which sought to analyze language skills needed in the logistics area.

This situation shows the scarcity of research investigating the expected English language skills from graduate students in Brazil. However, lato sensu postgraduate courses, mainly from public institutions, select their candidates through admission processes which, among the various instruments, use written tests that cover different areas of knowledge, such as English language. Nevertheless, to ensure that the test is valid, an accurate description of the expected level of proficiency is required to achieve the objective of its use fairly to the candidates.

The validity of a test in language assessment presents several aspects, such as content, construction, criteria and face. However, for this investigation, the validity of content is seen as the most important. For Hughes (1989), a test is considered valid if it presents a representative sample of the language skills and relevant structures that are intended to be tested. Therefore, in order to judge the validity of a test, specifications of the skills or structures that the test is intended to cover are required. Hence, the comparison between the content of the test and its specifications will be the basis for judging the content validity (HUGHES, 1989).

Aside from the validity aspect of the test, it is essential to emphasize the effect that a test can have both on the candidate's life and on society, which may involve getting a job, a scholarship, a place in university. Bachman (1990) points out that the most important consideration both in developing tests and in interpreting their results is the purpose or purposes that the tests should serve. This is because tests serve the educational system or society in general, that is, their consequences are not isolated. This situation demonstrates that tests are based on decisions that affect the candidates’ life and this perception can vary according to the perception of the exam relevance (MADAUS, 1988).

3. METHODOLOGY

This study is a qualitative investigation which according to Denzin and Lincoln (2006) consists of an activity that seeks to understand the phenomena in terms of the meanings that people give them. Furthermore, according to the authors, the emphasis of this type of research is on the processes and meanings that are examined or measured experimentally.

To accomplish this, the data obtained from different sources are triangulated.

triangulation is not a validation tool or strategy, but an alternative to validation. The best way, then, to understand the combination of a multiplicity of methodological practices, empirical materials, perspectives and observers in a single study (FLICK, 1998, p.230- 231)

Data triangulation may be from sources, methods researches and theories. For this study, data collected from different sources such as documents, tests and questionnaires are triangulated.

3.1. Scenario

The postgraduate course in Logistics and Operations has a workload of 400 hours and is offered by a public institution located in the Great São Paulo. The admission test is composed of 40 multiple choice questions divided in four areas of knowledge: English, Portuguese, Mathematics and Logistics. The admission process occurs twice a year and there are 50 available vacancies annually for graduates in any field.

3.2. Instruments

Firstly, the documents that guide the postgraduate course in Logistics were analyzed to infer the language skills required for the course. According to Alves-Mazotti and Gewandsznajder (1999), document analysis can be used as an exploratory technique as well as for checking or complementing the data collected from other instruments. The five documents below rule the courses offered by the entire federal system of higher education.

a) The resolution CNE/CES nº 1, from April 6, 2018 brings guidelines and standards for Lato Sensu postgraduate courses in the federal system of higher education;

b) The resolution nº 64, from August 1, 2017 approves the regulation of the postgraduate course offered by the institution;

c) The normative nº 1/2017 contains information about the implementation, update, reformulation, temporary interruption or extinction of the postgraduate courses offered by the institution;

d) The normative ruling PRP nº 1/2018 presents information about distance learning on the postgraduate courses of the institution and;

e) The Pedagogical Project of the Lato Sensu course in Logistics and Operations (PPC) presents the course objectives, conceptions and guidelines.

The questionnaire for the program faculty aimed to outline the language skills required from the students to attend the professors’ lectures. A questionnaire was also applied to the course coordinator in order to identify the objectives of the entrance examination and the English test.

The advantages of questionnaires are that the data themselves are easier to quantify than discursive methods such as transcripts of oral language (NUNAN,1992). The items can be closed, in other words, they present answers established by the researcher that must be elected by the participant, or open, that is, the answers must be given by the individual. Both types have advantages and disadvantages, as Nunan (1992) points out since responses to closed items are easier to collect and analyze while open items probably come closer to what the participant really means.

The questionnaire applied to the professors was composed of closed items while the one applied to the coordinator contained closed and open items. This tool proved to be more adequate due to the details and specificities of some questions.

Moreover, a statistical analysis was carried out of the second semester of 2018, first semester of 2019 and second half of 2019 admission tests scores. The aim was to investigate the role of the English test in the admission process.

3.3. The admission process

The admission process is carried out in a single phase consisting of an objective test divided in the four mentioned areas. Each one has a specific weight in the final score. To be approved the applicant must score at least three questions in each area.

Table 1: Admission test for the lato sensu postgraduate course in Logistics and Operations

Area	Number of questions	Weight	Required number of correct questions
English	10	2,0	3
Mathematics	10	3,5	3
Portuguese	10	2,5	3
Specific knowledge	10	2,0	3

3.4. The professors and the coordinator

The faculty is composed of six masters and six doctors who teach the fifteen disciplines set in the PPC like Standards and Regulations, Leadership and Human Capital Development, Production Management, Operations Management, Supply Chain Management among others. The professors’ experience encompasses work in the academic area as well as in national and multinational companies. The coordinator is also a professor in the course and has experience in the areas of international logistics, foreign trade and agricultural engineering.

4. RESULTS

In this section the data collected with the questionnaires to the professors and the coordinator is analyzed as well as the program documents and the previous admission tests results.

4.1. Questionnaires to the professors

When asked about the activities Logistics and Operations professionals should perform in English to meet the job market demands, the most mentioned answers by the professors were to read reports from the company where they work and read external e-mails. They also mentioned: receiving clients, suppliers and foreign peers; attending presentations, conferences and other events and calling clients, suppliers and peers in other countries. While the less mentioned alternatives were: writing reports; reading newspaper articles to follow the market and making presentations. The graph presents the number of mentions for each alternative of the questionnaire.

Figure 1: Activities student should perform in English for the job market

Regarding what students must read in English in order to follow the professors’ discipline, all participants mentioned paper reading. While reading manuals, excerpts from articles, software in English and books were not mentioned or were mentioned only once, as the following chart exemplifies.

Figure 2: Documents students need to read in English

As for the written production skills, the most of the professors cited abstract writing for event submissions, while writing forms, reports and projects were the least cited alternatives.

Figure 3: Documents students need to write in English

Asked if speaking skill is required in their discipline, only two professors claimed that it would be speaking moments in company visits, congresses, fairs etc.

Figure 4: Opportunities for student to speak or listen to English language

Finally, six English level descriptions taken from the Common European Framework (CEF) were presented to the professors. The participants should choose the most suitable alternative to a Logistics postgraduate course student. The most quoted option was: "Can understand the main issues, when clear and standardized language is used and the subjects are familiar to him/her (subjects covered in the work, at school and during leisure time, etc.). Can deal with most of the situations found in the region where the target language is spoken. Can produce a simple and coherent discourse on topics that are familiar or of interest to you. . Can describe experiences and events, dreams, hopes and ambitions as well as how to briefly state reasons and justifications for an opinion or a project." Such a level is equivalent to B1.

The second option mentioned by the professors was: "Can understand the main ideas in complex texts on concrete and abstract topics, including technical discussions in its area of expertise. Can communicate with a certain degree of spontaneity to native speakers, without tension on either side. Can express itself in a clear and detailed manner on a wide variety of subjects and t explain a point of view on a current issue, exposing the advantages and the inconveniences of various possibilities". This description is equivalent to B2 level.

Figure 5: Language skills of the professional with a postgraduate diploma in Logistics

Accordingly, considering the specificities of the disciplines taught by each professor and the descriptions most pointed out, it is possible to state that the level required varies between B1 and B2.

4.2. Questionnaire to the coordinator

The questionnaire applied to the coordinator presented similar questions to the professors’ one. However, the focus was on the skills expected from the postgraduate student instead of one specific course subject. In addition, the tool had two open questions on the admission objective and the English test.

When asked about what students should perform in English to meet the requirements of the job market, the coordinator pointed out the following activities: reading reports from the company where he/she works; reading newspaper articles to follow the market; reading external e-mails; reading manuals; reading technical documents; writing e-mails; writing other documents; participating in meetings; calling clients, suppliers, peers etc. in other countries and attending presentations, conferences and other events. The only options not mentioned were: writing reports, making presentations, receiving clients, suppliers and foreign peers.

As for the reading skills student must have, the coordinator pointed out reading papers, printed and online media articles, book chapters and software instructions and application.

Regarding what the students need to write, the genres cited were: simple abstract, extended abstract and scientific articles. Projects, reports and forms were not mentioned.

About opportunities to speak and/or listen to English language in the course, the coordinator stated that there are chances in the lectures, videos or in visits, events such as congresses, fairs, etc.

The six levels that describe the foreign language user's abilities according to the CEF general descriptive scale were presented to the coordinator. The alternative chosen by the participant was the B2 level: "Can understand the main ideas in complex texts on concrete and abstract topics, including technical discussions in its area of expertise. Can communicate with a certain degree of spontaneity to native speakers, without tension on either side. Can express itself in a clear and detailed manner on a wide variety of subjects and t explain a point of view on a current issue, exposing the advantages and the inconveniences of various possibilities”.

Concerning the admission test objectives, the coordinator emphasized its diagnostic nature.

The objective test of the admission process is composed of four distinct components, English, mathematics, Portuguese and specific knowledge. The general objective of the test is to identify the candidates' skills regarding language abilities (comprehension, interpretation of text and grammar), mathematical logic (application of mathematical rules and problem solving) and specific knowledge in the area of logistics, field of work and its applications. At the same time, the test enables the identification of the main points of difficulties of the candidates in the various areas; therefore, it is easy to diagnose if the candidate has an easier development in certain activities that involve specific components.

When asked about the objective of the English exam for the postgraduate course, the coordinator answered:

Considering the dynamics of the current market inserted in an international context, the English language is simply essential for any activity, especially for logistics that connects global chains through their activities. Our Postgraduate Course in Logistics and Operations, among the various points of distinction, stands out for requiring the development of scientific papers as one of the alternatives for graduating, unlike most Lato Sensu postgraduate courses that no longer require the preparation of an end of course paper. This means that the student must go beyond the boundaries of language and knowledge and this is only possible with English. Thus, the English test applied in the admission process has as main objective to identify if the candidate has the necessary knowledge for the correct text comprehension, language understanding, vocabulary and English expressions use. The objective is to know if the candidate has the ability to apply the grammatical rules and the correct use of the language, that is, if he/she is proficient in English.

This answer shows the importance of the English language not only for the area of logistics but also for the development of technical-scientific knowledge, since the domain of English is essential for the development of academic works. It is observed that the language requirements are focused on the ability to understand texts in addition to mechanisms that are essential to the writing process such as vocabulary, language expressions and grammar, which are core to the development of activities such as scientific papers, which are required to finish the course.

4.3. Documents analysis

This section presents the most relevant data from the lato sensu postgraduate course's guiding documents in the federal educational system and the institution under this investigation.

Table 2: Documents that guide the postgraduate course in Logistics

DOCUMENT	PURPOSES
The resolution CNE/CES nº 1, from April 6, 2018	Establishes guidelines and norms for the offer of the lato sensu postgraduate courses denominated specialization courses, in the scope of the Federal System of Higher Education
The resolution nº 64, from August 1, 2017	Approves the regulations of the Lato Sensu postgraduate course of the institution analyzed
The normative ruling nº 1/2017	Provides clarification about the legal files and processes for implementation, updating, reformulation, temporary interruption and termination of courses and postgraduate programs - lato and stricto sensu of the institution analyzed
The normative ruling PRP nº 1/2018	Sets the criteria for organizing lato sensu and stricto sensu postgraduate courses and programs in the modality of distance education
The Pedagogical Project of the Lato Sensu course in Logistics and Operations (PPC)	Introduces the purposes, concepts and guidelines for the operation of the lato sensu postgraduate course in Logistics and Operations

The normative instructions no. 1/2017 and PRP no. 1/2018 present organizational guidelines, such as the course implementation process, criteria for the course offer and rules for distance learning courses. However, they do not present information related to the entry test specifications.

The CNE/CES Resolution no. 1, of April 6, 2018, brings in its first article, the definition of postgraduate courses, establishing as objectives of the programs the academic improvement, the incorporation of new technical skills and professional profiles in order to promote the development of the country.

The Resolution no. 64 of August 1st, 2017 presents more relevant aspects for this investigation. In its article 2, the document states that the objective of the Lato Sensu postgraduate courses is "develop specific activities in research and teaching, aiming at the preparation and specialization of professionals with higher education for academic activities in different fields of knowledge, enabling the expansion of technical competence and the improvement of studies in the various areas of knowledge".

In addition, Article 3 of the document characterizes postgraduate courses as instruments for complementing and expanding theoretical and practical knowledge in a particular area. The 5th article establishes the cultural, scientific and technological education, the qualification for the job market, the integration between the teaching levels and the critical commitment with the development of the country as principles to be observed in the implementation of the courses. Article 10 of the resolution states that postgraduate courses may promote exchanges and partnerships with other educational institutions.

In all these articles, it is possible to infer the student's need to act in areas where English is the dominant language, such as research, science, technology, in addition to taking part in internationalization actions, which reaffirms the need for the requirement of the English language in postgraduate courses.

On the other hand, the pedagogical project of the course (PPC) brings information about the educational institution, the justification of the project and the details of the lato sensu postgraduate course in Logistics and Operations. By presenting the general details of the educational institution, the PPC establishes the mission of the institution to consolidate an educational practice that promotes social insertion, knowledge production and social education.

On the topic Justification and Market Demand, the document highlights: " (...) the professional training in Alto Tietê in order to enable students to work in marketing and distribution of logistics products, planning, driving, monitoring, managing and controlling processes, products, and productive factors used in the regional supply chain".

The objective of the course is to train professionals capable of working in the production chain, in order to consolidate them in the region, besides introducing students to research. Among its goals are: technological quality of logistic processes, professional training, scientific research promotion in the region and scientific and technical knowledge promotion through papers, presentations in congresses, field days and technical production.

Once again, in a similar way to what was established in Resolution 64, of 01 of August 2017, there is an emphasis in the PPC on acting in the same areas cited in the resolution. In addition, the document focuses on regional development, an area which hosts several multinational companies which reinforces the relevance of English language in the course. This becomes more evident when the PPC sets a paper written in a foreign language as one of the end of course possible work.

4.4. Past admission tests

This section addresses the results of the last three admission tests for the postgraduate course in Logistics. It describes the scores achieved by the applicants in the four areas covered in the test.

In the second half of 2018 test, there were 125 candidates to fill the 30 available vacancies. The first analysis aimed to identify the components with the lowest scores among the participants. In cases where the candidate had the lowest scores in two subjects, both were considered. The following table shows how many applicants got their lowest scores in English, mathematics, Portuguese or specific knowledge areas.

Table 3: Areas in which applicants got the lowest scores (2nd semester of 2018)

2nd semester of 2018 admission test
Area	Number of candidates who got the lowest scores
English	66
Mathematics	50
Portuguese	27
Specific knowledge	30

The area in which most candidates had their lowest score was English, followed by mathematics.

Figure 6: Number of applicants who failed in each area

In the following admission process, the English language was again the component with the highest number of lower grades, followed by specific knowledge and mathematics as shown in the table below.

Table 4: Areas in which applicants who got the lowest score (1st half of 2019)

1st semester of 2019 admission test
Area	Number of candidates who got the lowest score
English	80
Mathematics	44
Portuguese	35
Specific knowledge	52

Again, English had the highest number of applicants who did not get the minimum score while Portuguese language remained the area with the lowest failed candidates. The figure below shows the number of candidates eliminated in each area.

Figure 7: Number of applicants who failed in each area (1st half of 2019)

In the last admission test, the number of candidates with the lowest score in English remained the highest while Portuguese kept the lowest numbers. The number of students with the lowest score in specific knowledge exceeded the mathematics area as shown in the following table.

Table 5: Areas in which applicants who got the lowest score (2nd half of 2019)

2^nd semester of 2019 admission test
Area	Number of candidates who got the lowest score
English	93
Mathematics	64
Portuguese	38
Specific knowledge	91

Concerning the number of candidates eliminated in each area, the previous results remained the same. However, in this admission test the lowest scores and failures in specific knowledge area increased. The following graph shows the number of participants who failed in each area.

Figure 8: Number of applicants who failed in each area (2nd half of 2019)

Based on this data analysis, it is possible to see that the English test has been an obstacle for some participants which may be consequence of several factors such as issues in English learning and teaching, problems in preparing for the admission test among others.

5. DISCUSSION

The English language relevance in the postgraduate course in Logistics and Operations was identified in this study. First, it was possible to recognize the importance of scientific research in two documents: in the CNE/CES resolutions no. 1, of April 6, 2018 and no. 64, of August 1, 2017, as well as in the PPC. Although English was not explicitly mentioned in both, it was underlying since the development of academic work requires reading seminal and current papers, books and journals that are mostly not written in Portuguese, but mainly in English, since academic publications are published in this language.

This result was confirmed by the professors and the coordinator, when it is claimed that students need to read and produce academic papers. In addition, it was also pointed out the importance of writing simple abstracts in English to be submitted in academic events.

The importance of trained professionals for the labor market is also present in CNE/CES resolutions No. 1 of April 6, 2018 and No. 64 of August 1, 2017, and in the course's pedagogical project. The documents state that postgraduate courses must contribute to qualify students by incorporating new technical knowledge and enabling the emergence of new professional profiles.

These requirements may encompass a series of skills but due to the "new technical knowledge" English language comes as fundamental, since similarly to what was discussed about scientific knowledge, an updated professional means being aware of what happens in the globalized market and not only restricted to the domestic market. In addition, "new profiles" now covers also live and work in this globalized world which is characterized by English as a lingua franca.

Furthermore, in the questionnaire applied to the professors and the coordinator, the activities considered most relevant were reading reports from the company where he/she works and external emails, receiving and calling clients, attending presentations and writing emails in English. Again, without the necessary language knowledge to carry out such activities, the students’ performance may be hampered.

Moreover, the Resolution 64 of August 1, 2017 and the PPC bring aspects about the critical education and commitment to the country development as the course objectives. These goals can be related to English language as the development of a critical citizen in globalized times is only possible through the access to information as well as independent and effective actions. English allows the citizens to have these resources because it is the main language in both scientific publications and media.

The statistical analysis of the admission tests results show what has already been discussed by Leffa (2005) and Paiva (2011) since issues in English language proficiency of Brazilian students have also been identified.

Based on the data collected through the various sources it is possible to establish the specifications of skills and structures that should be required in the entry test for the postgraduate course in Logistics. According to Hughes (1989), a test is valid if it presents a representative sample of language skills and relevant structures that are intended to assess, and in this study the skills are related to reading specific genres like papers and media publications, as well as writing genres like abstracts and if the students want papers as one option to the work to be presented at the end-of-course requirement.

In addition, the data analysis show the English language level required for the postgraduate course admission should be CEF B2 considering some professors and the coordinator participation in this investigation. However it was not a clear-cut as B1 level was also chosen by most of the professors. Therefore, it is safe to say that B2 level students would be better able to deal with more complex and abstract texts which usually circulates in the academic and professional world.

6. CONCLUSION

This investigation aimed to infer the English language competences required from a Logistics and Operations postgraduate course applicants. The analysis of documents and questionnaires applied to the course professors and coordinator showed the importance of English not only for the activities linked to the course but also to the labor market, technical-scientific development and critical thinking of the student. Moreover, the professors’ expectations on English use indicate that the admission test must assess the reading skill of academic and professional texts, reflecting the description of the CEF B2 level, and the writing skill to produce abstracts.

REFERENCES

ALVES-MAZZOTTI, A. J.; GEWANDSZNAJDER, F. (1999) O método nas ciências naturais e sociais: pesquisa quantitativa e qualitativa. 2. ed. São Paulo: Pioneira.

BACHMAN, L. (1990) Fundamental Considerations in Language Testing. Oxford: Oxford University Press.

BRASIL (2017) A internacionalização na Universidade Brasileira: resultado do questionário aplicado pela Capes. Brasília: Ministério da Educação, 51 p.

BRASIL (2016) Catálogo Nacional de Cursos Superiores de Tecnologia. Brasília: Ministério da Educação, 3 ed.

BRASIL (2017) Censo da educação superior. Instituto Nacional de Estudos e Pesquisas. Diretoria de Estatísticas e Avaliação da Educação. Brasília: Ministério da Educação.

CONSELHO NACIONAL DE EDUCAÇÃO (2018) Resolução CNE/CES nº 1, de 6 de abril de 2018. Diário Oficial da União, 9 de abril de 2018, Seção 1, p. 43.

DENZIN, N. K. (1978) Sociological methods. Nova York: McGraw-Hill.

DENZIN, N.; LINCOLN, Y. O (2006) Planejamento da pesquisa qualitativa: teorias e práticas. Tradução Sandra Regina. Netz. Porto Alegre: Armed.

FLICK, U. (1998) An introduction to qualitative research: Theory, method and applications. London: Sage.

GRADDOL, D. (2000) The future of English? The British Council.

HUGHES, A. (1989) Testing for language teachers. Cambridge: Cambridge University Press.

INSTITUTO FEDERAL DE SÃO PAULO (2017) Instrução Normativa nº 1/2017. Accessed at https://prp.ifsp.edu.br/images/Instru%C3%A7%C3%A3o_normativa_-_01-2017.pdf.

INSTITUTO FEDERAL DE SÃO PAULO (2017) Resolução nº 64, de 01 de agosto de 2017. Accessed at https://prp.ifsp.edu.br/images/Resolu%C3%A7%C3%A3o_64_-_2017_novo_Regulamento_para_PG_Lato_Sensu_copiar.pdf.

INSTITUTO FEDERAL DE SÃO PAULO (2018) Instrução Normativa PRP nº 1/2018. Accessed at https://prp.ifsp.edu.br/images/DPG/Documentos/Intru%C3%A7%C3%A3o_Normativa_PRP_n%C2%BA_1-2018_-_EaD_na_p%C3%B3s-gradua%C3%A7%C3%A3o.pdf.

JOHNSTON, M. (1985) Syntactic and Morphological Progression in Learning English. Canberra: Department of Immigration and Ethnic Affairs.

KOBAYASHI, E.; GALLARDO, B. (2018) Cambridge English tests to assess proficiency of Letters program students: an investigation. Relatório de pesquisa, Universidade de Cambridge.

LANKSHEAR, C. (1997) Language and the new capitalism. The International Journal of Inclusive Education, v.1(4): 309-321.

LEFFA, V. J. (2005) O professor de línguas estrangeiras: do corpo mole ao corpo dócil. In: FREIRE, M. M.; ABRAHÃO, M. H. V.; BARCELOS, A. M. F. (Org.) Lingüística Aplicada e contemporaneidade. São Paulo: ALAB; Ponte, p. 203-218.

MADAUS, G. (1988) The influence of testing on the curriculum. In: TANNER, L. N. (ed). Critical issues in curriculum. Chicago: Chicago University Press, p. 83- 11.

NUNAN, D. (1992) Research Methods in Language Learning. Cambridge: Cambridge.

PAIVA, V. L. M. O. (2011) Ilusão, aquisição ou participação? In: LIMA, D. C. (Org.). Inglês em escolas públicas não funciona: uma questão, múltiplos olhares. 33-46.

VALETTE, R. M. (1977) Modern Language Testing. New York: Harcourt Brace Jovanovich.