Activity 3

 Validity

It is said that a test has Validity when the content represents a sample of the language skills, structure, etc. It is important to have in mind that there are two kinds of validation, which are: concurrent validity and predictive validity. The first takes time when the test and the criterion are administered at the same time. and the second one is concerned with the degree to which a test can predict candidates future performance. 

It is important to mention that content validation confirm that the sub-skills are well represented in the test. There are two methods used to gather information that demonstrates the evidence of the construct validity, and those are: the think aloud, in which test takers voice their thoughts as they respond. And the retrospective method in which test takers try to recollect what their thinking was as they responded. A valid test provide consistently accurate measurements. 

Reliability: 

The term reliability on a test refers to the different results we get while applying the test, since we know that the scores could differed if we apply the test in different circumstances. The reliability coefficient allows to compare the reliability of the test. However, it does not provide directly information of how close the candidate score is to what the candidate has scored in other occasions. It is important to take into consideration that to make a test reliable, we must consider the performance of candidates from ovation to ovation, and the reliability of the scoring. 

Backwash: 

The term backwash refers to the effect that tests have on learning and teaching, meaning that both are linked. It is necessary either for students and teachers to understand the tests demands in order for the effect to be fully realized. It is important to have informed, prepared teachers to avoid chaos and dissatisfaction, because tests are meant to help change teaching, and they need support to help effect the change. 

Analysis of a Test

1. What is the purpose of the test?

To evaluate level 4^th students about their reading writing, listening and mainly grammar (There were several exercises to assess grammar)

2. Does it represent direct or indirect testing or a mixture of both?

It represents mainly direct testing because it is assessing the skills by performing them

3. Are the items discrete point, integrated or a mixture of both?

The test included both. The only integrated item was the writing, the others are discrete. They assess one thing in each exercise.

4. Which items are objective, and which are subjective? Can you order the subjective items according to the degree of subjectivity?

The only subjective activity is the paragraph writing.  The others are objective. There is only one possible answer.

5. Is the test norm-referenced or criterion referenced?

It is criterion referenced because the exercises are assessed according to some established criteria and  they are not assessed to compare the students.

6. Does the test measure communicative abilities? Would you describe it as a communicative test?

In a sense we might say that some exercises look communicative, specially the writing exercise, the reading and listening. However, the grammar ones are very artificial and do not look like a real life activities, they are very mechanical. (Some of you said yes but the answer was not so clear)

7. What relationship is there between the answers to question 6 and the answers to the other questions?

The relationship between answer 6 and the others is that communicative tests tend to include direct testing, the items are integrative and subjective because acts of communication and unpredicted and integrate two or three skills and different types of knowledge

Types of Test and Testing

  

Types of test

Proficiency test: 

This type of test shows the capacity of a student in having sufficient command of the language for a particular purpose. in other words, it measures the student’s knowledge on the language. Proficiency test might be general or specific, for example, IELTs is specific because it measures academic language and TOEFL is more general although it is usually a requirement to enter an academic program.

Progress/Achievement test: 

Is the type of test that takes place throughout the course, it provides information to improve knowledge. It is a good source of feedback, since depending on the results the teacher can know the student’s strong and weakness measuring the students progress on the language. An example of progress test could be quizzes. 

Summative/Final test: 

This test is made at the end of the course to see if students have achieved the goal set in the syllabus. It is based directly on the detailed information of the course and it does not provide feedback. 

Diagnostic test: 

It is used to analyze students strong and weaknesses, mainly to be able to improve. It takes place at the beginning of the course or unit. The test made on the first day of school could be an example of this kind of test. 

Placement: 

takes place at the beginning of the course and they are meant to provide the information to place the student on a certain level depending on his/her abilities or capacities related to the language. Example of this is the test we made when entering a language school, they made us a test to see in which level/class we will enroll. 

Types of testing

 Direct versus indirect testing: 

Direct testing is made when we need the student to perform a skill in specific that we want to measure. For example when we need to measure oral skills then we ask the students to produce an oral conversation. In the other hand, Indirect testing helps us to measure an ability in specific in an indirect way. For example if we need to asses grammar we will ask a student to perform an activity that involves grammar such as writing a text or producing some oral activity, this way we will assess grammar but we are not asking our students to perform grammar specifically. 

Discrete point versus integrate testing: 

Discrete point testing makes reference of testing “one element at a time, item by item”, knowing that this takes the form of series of series, and each testing on that serie test a particular part that we aim to measure. While integrative testing suggests the student to to combine many different elements to complete a task, for example writing a composition, in which the student will use vocabulary, punctuation and grammar in order to complete the task.

 Norm-reference versus criterion-referenced testing:

This kinds of testing refer to the results of the performing of students after a test, this could be separated into: non-reference, that mainly relates to the students’ performance comparing it with other students. Meaning that we do not have direct information related to what the student knows or is capable to perform in the language. While the purpose of criterion-referenced is to classify the students depending on what they are able to perform satisfactorily on a language. 

Objective testing versus subjective testing:

This types of testing refer completely to scoring, the difference is that in the objective testing there is no judgement required on the part of the scorer, this is kind of test are also known as one possible answer only, and that is mainly the resin they do not need to be judged. While when there is judgment is called subjective. here in the subjective testing we have different degrees of difficulty, this type of testing are known as short answers in which the student will answer in a wide variety of ways depending on the style and level in each student. 

Computer adaptive testing: 

This kind of testing provides an efficient way for gathering information on student’s abilities. in here students are tested with average items if they answer correctly they will be tested with a more difficult/advanced one, while those students who answered incorrectly they are tested with easier items. In this process, the commuter keeps on going until comes up with the students apparent level of ability in the language, or at least an estimate. 

Communicative Language Testing: 

This kind of testing is centered in the ability of the student to communicate and take part in acts of communication. 

Reflection Unit 2

When I first started this course, I though assessment was going to be a difficult subject, specially because I thought it was going to be hard to differentiate between assess and test. However, while the course goes by, this terms become easy to understand and I like that i become more involved with them and I can't wait to use them in my classes!

Through this unit I learned that “Informal Assessment” gives an idea of being a “non reliable” or unattended assessment, but this idea is so far from reality since Informal Assessment is a system of observation and collection of data, and not merely the teacher’s opinion about students’ performance. So in order to carry out an effective informal assessment, the teacher has to decide what he/she wants to assess, taking into consideration both linguistic and non-linguistics aspects present in a language class environment. Once the teacher knows what to assess, a clear criteria for assessing must be established, this way the same criteria will be applied for all the students in a normal class situation. An effective way to do this is by creating bands or rubrics, which can help the teacher to know what the students are expected to achieve in a certain activity, as well as what grades or points the students are going to get according to the level they reach during performing that activity.

It was interesting to know that creating rubrics is not an easy task since the teacher has to establish from the beginning the performance goals of the activities that the students do at classroom. In other words, the teacher has to let know the students what is the best way of demonstrating competence in a given activity, and obviously, what is the worst way to do it. The teacher needs to select those activities carefully, so they can be included in the assessment plan either informally or formally and focused on helping students to develop the expected competencies daily.

Activities and instruments to assess informally

Oral production

As my oral production activity we will play a game called “I need a job” in which first we will create a personal resume, after Ss have their own resume we will arrange teams and we will pretend to be looking for a job. I will divide the group in two groups. Group number one will be the employers and team number two will be the ones looking for a job. Team 1 will be divided in 3 teams and each team will have to create a company that is hiring new personal. Ss will have to interview for different positions, meaning that they will be interviewing team number 2 in search of their perfect employees. For this activity they will use communication to decide what kind of company they want to create as well as during the interview they will be able to ask the abilities, occupations and obligations of the person they will be interviewing (in case of team 1) and they will express their abilities, occupations and necessities (in case of team 2).  Ts will provide each company with the different positions they will be looking for, so each team will be looking for different employees giving them the chance to practice different occupations and abilities as well as obligations. (20%)

Writing

 For writing, We will be playing a game called “you’ve got mail” for this game, I will divide the group in half. we will be pretending to have a huge important party. and each team will represent a role. Team one will be the ones inviting to the party to team 2. Team two will have to respond that letter with an apology/giving reasons why they will or will not attend the party. I will take a huge mail box to class. Starting with team 1 they will have 5 minutes to write down their invitation explaining why they want the other person to join the party. They will put all the letters in the mail box and team 2 will come to the mail box and choose a letter randomly, they will read the content of the letter and they will have to answer to that letter, having to use an apology or given reasons. They will also put all the letters in the mail box and team 1 will come to the mail box pick a letter randomly. In the second stage of the activity each student will provide feedback for their classmates. For this, I will provide the paper sheets they will be using for the activity, and each piece of pepper will be numbered to keep a track of who wrote each letter, so at the beginning on the class I will give a number to each Ss from 1-20 the pairs will be team 1 the nones will be team 2 to make sure they all have a number. After they all provide feedback to the letter they got, they will all put the letters with the feedback in the mailbox. I will review each of the letters and I will assess grammar, vocabulary and punctuation to each of my students from their own writing and the feedback given. (10%)

Reading

For reading, I will provide my students with an article about obligations. First they will scan the article looking to have an idea of the article given. Secondly, they will skim the article in search for certain obligations in the article and they will underline them. and finally they will be given a fill in the blank activity in which they will use the information they read in the article. (5%)

Listening

As a wrap up activity I will play an audio to the class called “who am I?” the audio basically activities related to different occupations. I will provide Ss with sheet that contains a series of occupations in which after I play the activities and the students listen to them, they will have to identify the occupation  in the sheet I provide to them according to the activities they heard. In this activity Ss will develop listening skills and I will have the chance to assess how much they understand by listening. (5%)

Oral Production: 

Grammar	Clarity	Understanding	Vocabulary	Effort
Uses grammar structure correctly with few or no mistakes at all. 4 points	Message is easy to understand.  4 points	Comprehends without difficulty.  4 points	Uses vocabulary effectively. 4 points	Willing to participate and enroles in the activity.  4 points
Has a little difficult with grammar structure and a few mistakes 3 points	message understandable but has some mistakes and sometimes forgets vocabulary.  3 points	Comprehends almost everything and sometimes has some difficulty.  3 points	Uses basic ans learned vocabulary.  3 points	Has desire to participate but has some mistakes which makes him/her not too active in the activity. 3 points
Numerous  mistakes with basic structures. Sometimes the meaning is not clear. 2 points	Message is understandable for basic information. Makes a huge effort to communicate, sometimes needs help with vocabulary 2 points	Shows confusion and struggles to understand 2 points	Has a limited/basic vocabulary. 2 points	Makes some effort but uses mother tonge to ask for help.  2 points
Lots of mistakes with grammar structure meaning is hard to understand.  1 point	Very difficult to understand, Hesitates a lot, forgets vocabulary needs lots of help and patience of the listener.  1 point	Unable to follow a simple conversation shows confusion and frustration.  1 point	has a poor vocabulary and it is imposible to understand because of the insufficient vocabulary.  1 point	Makes no effort to participate in the activity 1 point

Writing: 

Grammar structure	Vocabulary	Organization	Punctuation
Uses grammar correctly with few or no mistakes.  3 points	Uses a rich vocabulary at this level 3 points	Well organized, coherent, clear expression, easy to understand with few or no mistakes.  2 points	Uses capital letters, comas, periods. which makes the text easy to understand with few or no mistakes. 2 points
Has some mistakes and struggles a little with complex structures.  2 points	Uses basic and learned vocabulary correctly with few mistakes  2 points	Some problems with coherence but still understandable.  1 point	Miss some comas and periods but text is still understandable.  1 point
Has major mistakes with basic structures making the text almost imposible to understand.  1 point	Has a limited vocabulary which makes text hard to comprehend.  1 point	Disorganized, no coherence and really difficult to understand.  0 points	Text lacks of periods, comas and capital letters.  0 points

reading: 

Comprehension	Interpretation
Comprehends the text and has no difficulty finding information required.  3 points	Interprets the information accurately. 2 points
Comprehends most of the information but struggles finding specific information.  2 points	Has some difficulty interpreting the information given.  1 point
Struggles comprehending and finding specific information. Takes lots of time to read the text.  1 point	Unable to interpret information given. 0 points

Listening: 

Attention	identify main ideas
Payed attention throughout all the activity 2 points	Identified main ideas during the activity.  3 points
Was distracted at some points.  1 point	Has some difficulty to find main ideas and understanding the activity. 2 points
Did not pay attention or showed interest in the activity 0 points	Unable to understand the audio nor instructions. 0 points

Assessment Plan

 

In the present Assessment Plan we will assess linguistic and non-linguistic factors in a classroom situation; evaluating all four skills: speaking, listening, reading and writing.

Unit 13 Looking for a Job

General objectives of this unit: 

By the end of the unit students will be able to talk about their abilities, what they can or can not do; their occupations and obligations, what they have or do not have to do. They will also be able to apologize, give reasons and to invite.

Linguistics aspects: 

Can/Can’t Have to /don’t have to

Expressing ability

occupations

looking for a job

expressing obligation 

Grammar: test and activities

-Expressing abilities

Explain students the use of can/can’t to express their abilities using examples/whiteboard/textbook.

Make and answer questions using can/can’t when they ask and when they answer.

-Expressing occupations

Review occupations 

Review the use of can/can’t using occupations 

-Looking for a job

Review abilities, occupations, the use of can/can’t 

Practice occupations and abilities pretending to look for a job

-Expressing Obligation

Explain Ss the use of have to/don’t have to

Practice apologizing, inviting and giving reasons

Vocabulary: Test and informal assessment

Spelling:

abilities

occupations

apologizing

inviting

giving reasons

Speaking:

fluency

accuracy

vocabulary

Informal assessment:

talk about their abilities what they can or can’t do

look for a job expressing their occupations 

invite someone to different events

give reasons

apologize

Listening:

listen for details 

listen to other Ss while practicing

Writing:

Spelling

punctuation

Use of capital letters

run on sentences

Grammar

Write about their abilities, occupations, an invitation, an apology card

Non linguistic aspects 

-Participation

-Team work

-individual work

-provide feedback to other Ss

-practice good enough and is willing to aport and collaborate with the class

What would be assessed informally? 

Speaking 10%

intonation 5%

Reading 5%

listening 5%

Participation 15%

Group work 10%

Writing 10%

What would be assessed formally? (with a test) 

Writing

Grammar

Vocabulary

listening

Reading

What would be the weight?

Informal 60

Speaking 20%

intonation 5%

Reading 10%

Listening 5%

Participation 10%

Group work 10%

Formal 40

Writing 15%

Grammar 15%

Vocabulary 10%

Informal Assessment

Informal assessment is a system of observation and collection of data about our students’ performance in normal classroom conditions. In our daily job as teachers, we can intuitively assess students’ performance when speaking, writing, reading or listening just by paying attention to their attitudes and their participation in class activities, but if we want to carry out an effective informal assessment we need to carefully carry out systematic observation. To do this, firstly we need to decide what we are going to assess (either linguistic or non linguistic factors), then we must establish clear criteria for assessing students and not only rely on rough impressions, and finally we have to link the informal assessment we do with formal assessment in order to give our students as much of objective information as we can, so they can use this information as a reliable feedback that helps them to think for themselves about their performance.

Reflection Unit 1

   
In this unit, we had the chance to learn what an assessment is, as well as to differentiate it from testing and teaching. It is important to differentiate this three terms, since it is easy to get confuse and we will be using them a lot in the classroom.

We also learned about the portfolio, how is it used in the classroom and how we can encourage our students to used. It is important to know how to use the portfolio in class because it is a tool that it will provide us with the information that our students or even ourselves learnt and acquired throughout the course, such as information, reflections, and activities that students developed.

In the other hand, we also had the opportunity to reflect about the fact that using assessments will help us to get information not just from the students, but from and for different sources. With the assessment audit we learned to distinguish who the assessment is for. Making it a little bit easier for us to understand the topic.

Finally, with learning objectives we learned to create objectives  or competences that we could use or implement in our assessment plans. Personally, I loved this activity because it set me on the teacher environment, and for a moment I could feel like a teacher, thinking on all the competences I want to work on my students. I think that using this type of activities helps us to grow as future teachers.

Learning Objectives

- Here you will find a context in which we implement an Assessment Plan.

Objective of unit 13 Looking for a Job

By the end of the unit students will be able to talk about their abilities, what they can or can not do; their occupations and obligations, what they have or do not have to do. They will also be able to apologize, give reasons and to invite.

Contents of grammar:

Use of can/can’t 

have to/ don’t have to

Vocabulary:

Occupations: nurse, doctor, firefighter, carpenter, plumber, dentist, teacher, student, engineer, writer, dancer.

Abilities: draw, cook, swipe, clean, read, sing, dance, write, 

Listening:

Type of text: Conversations

Skills: Listening for accuracy: listening for details and listening to other Ss while practicing

Speaking:

Accuracy to express their abilities and occupations

Ask and express usual and unusual activities

Be able to apologize, invite to an event, or expressing obligation 

Writing:

Grammar

Capital letters

punctuation 

run on sentences

paragraph writing

Intonation:

Accuracy to express abilities/usual/unusual occupations/obligations

The Assessment Audit

- Here you will find reasons to assess students

 

Assessment Audit

For: English class Level 1

Who is this learning assessment for?	What do they need the information for?	What kind of information do they need?
Students	-So they feel confident. -So they get to know the course. -So they know what they can learn. -So they can set goals they want to achieve.  -So they can provide feedback to each other.	-They need feedback -They need to feel they are learning so they won’t feel discouraged. -They need to feel some passion for the language. -They need to know why is important to take the course.
Students parents and family	-So they can support Ss and help Ss out in case of need. -So they can encourage Ss to keep it up with the hard work. -So they know the Ss progress. -So they are aware of the Ss needs, weaknesses and strong sides.	-They need feedback on the Ss progress so they can help out. -They need to see examples of what we do in class. -They need to know how the class would be graded.
Principal	-So he/she can provide resources for class. -So he/she is aware of the goals to be reach in the class. -So he/she knows the way the class is managing.	-They need to know the necessities of the class. -They need to know the kind of resources the class needs.
Me (teacher)	-So I know what my students previously know. -So I know my Ss needs -So I know how to work with each of my Ss	-I need to have feedback from my Ss -I need feedback/tips from colleagues. -I need information of previous classes. -I need a syllabus
School district	-So they can provide us with what we need in the classroom. -So they know we are reaching our goals.	-They need evidence of the Ss learning. -They need to know the class’ needs.