ahh
LOGIN / SIGN UP
Te Arapiki Ako
"Towards better teaching & learning"
 

FAQs - Educational


Comment on this item  
 
Add to your favourites
Remove from your favourites
Add a note on this item
Recommend to a friend
Comment on this item
Send to printer
Request a reminder of this item
Cancel a reminder of this item
Share |
Last updated 26 October 2012 15:29 by NZTecAdmin

A general selection of questions and answers relating to educational aspects the Assessment Tool has been collated below. To view a selection of questions and answers relating to Technical aspects, please click here.

For questions relating to specific tabs within the Assessment Tool, please refer to the relevant 'Frequently Asked Questions' links within the 'How to use the Assessment Tool' pages.

If you do not find the information you are looking for, please email your enquiry to us.

Q1. What is the Assessment Tool? What was it designed to do?

The Literacy and Numeracy for Adults Assessment Tool is an online adaptive tool primarily designed to provide robust and reliable information on the reading, writing, numeracy and vocabulary skills of adults. This information informs the development of learning interventions that match learners’ needs and strengthen their literacy and numeracy skills. The Assessment Tool also allows learners to track their progress over time and enable educators and organisations to report on the progress made by groups or cohorts of learners. Further overview and background information is available here.

Q2. What is the difference between the Youth assessment option and the Adult assessment option? If I have a mixed age class which assessment option should I use?

The Youth option is aimed at learners aged between 15 and 25 years of age. It was developed to improve engagement with younger learners who may not have been as familiar with the workplace contexts reflected in many Adult assessment items. As a result Youth assessments contain more 'everyday life' contexts. Both Youth and Adult assessment options can be used with learners of any age.

The deciding factor should be what assessment content do you believe will best engage your students.

Q3. Should the Assessment Tool be used for formative or summative assessment?

It can be used for either one; but in actual fact it’s not the assessment that’s formative or summative, it’s how and when the information from the assessment is used. The Assessment Tool can be used to provide information that is used formatively to guide teaching and learning. It can also be used to measure learning in a summative way.

Q4. What will happen to all the data being collected?

Data from the Assessment Tool will be used by educators to monitor progress of learners in programmes that receive funding for literacy and numeracy. This learner data will be useful for tertiary organisations as they plan appropriate learning programmes and monitor their effectiveness.

The national data set will provide the TEC with information about the needs and issues of particular cohorts of learners in different contexts. The data is also available for research purposes within certain boundaries. At no time can any individual or individual organisation be identified in the national data set.

Q5. What is the purpose of assessing learner progress at Step 6?

The purpose of assessing a learner initially is to identify their skills. However, a learner who is placed at Step 6 in an initial assessment will not need further assessments using the Assessment Tool.

Q6. ITOs do not deliver training, but they arrange training through employers, PTEs and ITPs. Assuming the PTEs and ITPs use the Assessment Tool for pre and post assessments, do ITOs also need to be using the Assessment Tool to assess learners? If so, why?

ITOs need to understand whether the training they arrange is appropriate for their learners' literacy, numeracy and vocabulary skills and knowledge. This is particularly for ITOs that arrange embedded training programmes. They can monitor whether these programmes are strengthening learners' literacy and numeracy skills.

To avoid over assessment it is possible for an ITP or PTE to view an assessment result completed for the ITO, with the learner's permission.

Q7. When administering the Assessment Tool, what information do I tell the learners to make it a meaningful assessment for them and understand how it relates to their learning?

It is the role of the educator to explain to the learner that this is not a pass or fail test. It is an assessment that will help educators to strengthen the learning programme. The Assessment Tool is designed for people whose literacy and numeracy skills are within the Learning Progressions. Learners who do not reach a minimum of Step 1 of the Learning Progressions should not be given a Reading Assessment. However, they are likely to benefit from the Vocabulary Assessment and can have the Numeracy Assessment questions read to them.

For learners whose literacy and numeracy skills are within the Learning Progressions Steps, an online adaptive assessment is likely to be the most appropriate assessment for a learner who is wary of being assessed. An adaptive assessment starts at about Step 2 of the Learning Progressions and adapts straightaway to the learners' results - that is, if a learner struggles to answer the first few questions correctly, the computer will choose easier questions for them.

The educators can support the learner's understanding by:

  • How they introduce them to the Assessment Tool, including to the computer knowledge required
  • Using the Assessment Tool as it is intended; as one measure of a learner's achievement, and with learners whose literacy and numeracy skills are within the Learning Progressions
  • Discuss the report - to highlight strengths and areas to work on and follow up in their classes on areas needing work
  • Leaving a useful length of time between assessments (not over-assessing).

Q8. Why are some items context free?

A number of items designed to assess numeracy knowledge are context free in the sense that they do not sit within an authentic situation. These include, for example, items that assess a learner’s knowledge of number facts or their understanding of place value. These are deliberately without context so that the educator can determine whether the learners have control over specific sets of knowledge. When recognized or displayed automatically in isolation (context free) this knowledge will be available for the learner to draw on in any context. Of course an assessment that is set to guide teaching and learning is meaningful in its own right: assessments are set for purposes and in places that are relevant to the people undertaking them.

A similar approach is taken to assessing components of reading and writing as outlined in the Starting Points Assessment Guide.

Q9. Can I use the Assessment Tool for screening and diagnostic purposes?

The Assessment Tool has been designed to find out where a learner sits on the progressions in reading, writing, numeracy and vocabulary. Information about strengths and weaknesses can provide starting points to isolate specific learning needs. The Assessment Tool will give information about responses to individual items that represent different progressions as well as an aggregated score. The information will be most useful for learners when learners and educators together discuss a learner’s results and plan next learning steps.

Educators might use a variety of screening tools, such as interviews, questionnaires, assessments and specific tasks to give a full picture of a new learner’s strengths and abilities.

TEC guidelines around the use of the Assessment Tool are clear that assessment results should not be used to inform access to study and/or work opportunities. See the Acceptable Use Agreement  for more information.

Q10. Can learners be linked up to a qualification if their results are good enough?

No.

Listening, speaking, reading, writing and numeracy demands are embedded in many of the tasks that adults need to undertake in real life across a wide variety of contexts.

The Learning Progressions are designed to assist in planning foundation programmes where improved outcomes for learners in literacy, language and numeracy are desired.

The progressions provide a framework and language that can be used to identify next steps or gaps for learners as they develop expertise in these important areas. They have not been designed to link up with any qualification.

Q11. What research has been conducted to evaluate the results if learners merely guess at the multi-choice options?

Each item used in the Assessment Tool was trialled before being included in the Tool for general use. Part of the analysis of trial data involved checking for evidence that learners were resorting to guessing on each item. Guessing can never be ruled out when selected response type items are used. As a response strategy it will generally not lead to success.

When a learner is sitting an assessment that is 'at their level' and are motivated to choose the answer they believe is best, guessing behaviours can be minimised. The online adaptive assessment provides a way of targeting questions at the learner’s level. During the development of a question every effort was made to ensure that correct answers did not stand out from the other available answers to a question.

Q12. Is the Assessment Tool more useful in a tertiary institution than a workplace?

The Assessment Tool is suitable for use by any organisation that works with adults to improve literacy and numeracy. This includes tertiary education organisations that deliver formal education with embedded literacy and numeracy, community organisations that deliver intensive literacy and numeracy programmes and workplaces that provide programmes to strengthen employees' literacy and numeracy skills.

Q13. What are the current TEC reassessment thresholds for Reading and Numeracy?

Learners who have presented at Step 4 or above in reading, writing or vocabulary do not need to sit any further assessments. For numeracy the threshold is Step 5 or above. Visit the TEC guidelines page to read more about assessment expectations and the rationale behind current thresholds. Updated December 2014.

For further information please contact the TEC Service Centre by email: servicecentre@tec.govt.nz or Tel: 0800 601 301

Q14. What is the difference between contextualised and specialised assessment?

Specialised assessments are contextualised assessments that are specific to a particular course and would be developed and used in the context of the qualification that the learners are completing. The contexts used in the Assessment Tool are more generic, but are still intended to be meaningful to an adult.

Q15. How long should a provider hold onto their paper-based Assessment Tool scripts?

Providers should keep their paper records for seven years, as per general information storage guidelines.

Judgement can be applied to shorten the storage period if the provider is confident that the results from their paper scripts have been correctly inputted into the Assessment Tool system. This is because the Assessment Tool data will be stored on TEC’s servers.

Providers will need to be responsible for ensuring learners agree with the accuracy of the Tool results that are entered into the Assessment Tool.

For example if a previous learner comes back and contests some of their past results, it may be prudent if the provider still had the relevant paperwork on hand to check. 

Q16. How can Learners get a result above the selected non-adaptive assessment range?

When a Learner gets all or nearly all of the questions in a non-adaptive assessment correct they will be located above the scale locations of all or most of the questions in the assessment. This is because it is most probable that the Learner is capable of answering many questions that are more difficult than the ones that were administered. This can happen for instance, when a Learner does very well on an assessment targeted at Steps 1 to 3. In order to achieve this rate of success, the Learner is likely to be at Step 4 or above.


It is important to remember that when this happens the shaded plus or minus range associated with the Learner's score will be relatively large, sometimes covering two or more steps. This means the result is imprecise and that a second assessment could provide a different result. If an Educator wants a more precise result they will need to use an assessment where the Learner is likely to get some questions correct and some incorrect.

Q17. Why is there no ‘don’t know’ option for Learners?

The analysis that runs behind the responses a Learner makes is designed to accommodate instances where a Learner appears to have guessed an answer – this might be reflected by a Learner getting an item correct that all prior data suggests they should have got incorrect. A great deal of trialling and analysis has gone into the design of the current response options. The inclusion of a ’don’t know’ option could encourage students not to attempt items, irrespective of whether they find them challenging or not. We need as much information from the student as possible on which to estimate their ability.

Q18. The Learner reports provide an estimated best fit across the Steps (eg Reading). How does this reporting relate to spiky profiles, the range of reading comprehension strategies and range of vocabulary strategies?

The reports locate a Learner's achievement on the adult learning progressions. They provide a solid starting point for further exploration to identify particular strengths and weaknesses.


This starting point also provides a benchmark which can be used when looking at whether change has occurred over time. The potential of the reports doesn't finish there however - the reports can also be used to look at patterns of responses across individual progressions.


It is always important not to make too much of answers to individual questions. Educators should probe further to confirm any patterns they identify when examining a report for an individual.

Q19. Help! I'm still stuck!

Contact our Service Desk on (04) 381 4576 between 8am and 4:30 pm weekdays (closed public holidays and weekends) or email at any time assessforadults@nzcer.org.nz

 

Quicklinks

 

Comments

If you have any comments please contact us.

Search this section

Knowing the Demands Knowing the Learner Knowing the What to Do

News feeds

Subscribe to newsletter