Paper 39 – A digital assessment at Norwegian Universities
Transcript of Paper 39 – A digital assessment at Norwegian Universities
![Page 1: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/1.jpg)
Digital assessment A National coordination project
Freddy Barstad, National coordinator Digital Assessment, eCampus UNINETT
Bernt Smilden, Special Advisor, The Norwegian Business School
Agnethe Sidselrud, Deputy Manager - FS Consortium
EUNIS conference
Dundee, 2015-06-07
![Page 2: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/2.jpg)
Digital assessment as a whole
25. juni 2015 SLIDE 5
Digital assessment for all
TechnologyUNINETT
Pedagogical perspective
Organisational changeLegal issues
![Page 3: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/3.jpg)
Assessment process
6
Finalizing
Grading
Examination
Preperation
![Page 4: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/4.jpg)
Prepare
Examination
Grading
Feedback on
grade and
complaints
Locality
Candidate
Examination
/
questions
Medium
Answers
Status
Info
Journal
Digital
Manual
Digital Digital Digital
Manual Manual Manual
FS
Examination
-
registry
System for register
examinations
Formalities
Overviewt,e.g.
language-variants
Shortage, reminders
Archiving: Need of
history archiving.
Critical: Access
FS Answers
Systems for digital exam
Challenges:
•Control access
•Scaling
•Context
•Infrastructur
•Costs
•Competence
Critical: Stability
FS
System for digital
censorship
-Process support
-Transparent censorship
Support for varying
methodology
Critical: Access,
Security, Transfer
results.
In general:
•Must support digital og manualroutines
•Access to systems. People with loose connection to institution
•Digital signatures
•Clarify regulations
•Different forms of assessment
•Role of Learning Management Systems
•Hand over information to internal and national archives
Assessment process: From manual to digital routines
Need for system
support.
Case management
system
Integration with archive
system
Archiv
e
Inform the
students
Norwegian State
Education Loan Fund
Ministry of Education
(DBH)
Figure by NTNU
![Page 5: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/5.jpg)
What is Digital Assessment?
![Page 6: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/6.jpg)
Why Digital Assessment project?
Digital assessment is about working smarter, moving from paper based assessment
procedures to digital procedures, reducing the time and energy spent, and improving the
quality of the old written assessments procedures.
To change from paper-based procedures to digital procedures requires change in the
operations procedures at the university, the assessment regulations at the university
and the perception or understanding of the assessment processes, among both students
and staff at the university.
Students of today can’t write by hand anymore, they using PCs in their study, why can’t
they use PCs on exams?
There is no turnkey product, immature solutions from vendors, several promising
solution but lacking features supporting workflow.
![Page 7: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/7.jpg)
How is the Digital Assessment project
organized?
The digital assessment project, is a part of the eCampus program
UNINETT has established a national project to ensure that students get access to digital
examination.
All universities and university colleges was invited to participate in the project.
Created several working group, with participation from the universities.
Lots of coordination of existing activity
Facilitating exchange of experiences
Writhing specifications (CBP)
![Page 8: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/8.jpg)
Project organization
11
![Page 9: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/9.jpg)
Project deliveries
25. juni 2015 SLIDE 12
• Sharing knowledge
• Market development
• Pre-standardization descriptions (CBP-documents)
• Establish necessary infrastructure
• Prepare procurement(s)
![Page 10: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/10.jpg)
Sharing knowledge
25. juni 2015 SLIDE 13
Seminar series; https://www.uninett.no/digitaleksamen/arrangementer-om-
digital-eksamen
Web-site: https://www.uninett.no/digitaleksamen
Participation: https://www.uninett.no/digitaleksamen/nasjonal-
prosjektgruppe-digital-eksamen
Experience reporting: https://www.uninett.no/digitaleksamen/status-mars-2014
International collaboration jisc & GEANT & EUNIS(ELTF) & SUNET
![Page 11: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/11.jpg)
Market development
25. juni 2015 SLIDE 14
• Pre-commercial Procurement (PCP)
• Dialog conference
• Vendors day at seminar https://www.uninett.no/seminar-om-digital-eksamen/leverand%C3%B8rer
• Development agreements
• Sharing results
• Establish and testing integrations
![Page 12: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/12.jpg)
Pre-standardization descriptions –
CBP documents
The digital assessment market is immature, we use Current Best Practice (CBP) documents, and other reports as pre-standardization descriptions to generate commend understanding of needs and requirements for digital assessment.
CBP documents gives us a common understanding of digital assessment, common definitions, and a common starting point for future work.
CBP – Infrastructure (Norwegian / English)
CBP – Integrations (both in september 2015)
CBP – Client devices (Norwegian / English)
CBP – Digital assessment process & IT-architecture (Norwegian / translated to English in June 2015)
CBP - Logging and monitoring in digital assessment
EMA Toolbox?
![Page 13: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/13.jpg)
Workflow analysis
16
![Page 14: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/14.jpg)
Digital assessment
and
FS
Agnethe Sidselrud, Deputy Manager - FS Consortium
EUNIS conference
Dundee, 2015-06-11
![Page 15: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/15.jpg)
FS – Felles studentsystem
25. juni 2015 SLIDE 24
• National student information system in Norway
• Developed in cooperation with the HEIs, for the HEIs
• Organized as a consortium
• Mandate to standardize the processes within student
adminstration, to establish common procedures for the HE-
sector
![Page 16: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/16.jpg)
Tittel
25. juni 2015 SLIDE 25
• Tekst
![Page 17: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/17.jpg)
FS and Digital assessment
25. juni 2015 SLIDE 26
• Project in cooperation with Uninett
• Development of general integration with the digital
assessment systems
• General standards yet to be found – need for action plan
• In production spring 2014
![Page 18: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/18.jpg)
FS and Digital assessment
25. juni 2015 SLIDE 27
• Integration: stable, via national ESB
![Page 19: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/19.jpg)
FS and Digital assessment
25. juni 2015 SLIDE 28
• New functionality to come:
- Common systems for applying for explanation about grades,
appeal and complaints
- Web application for registration of grading
- Web application for registration of students’ participation
![Page 20: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/20.jpg)
Status for the FS-HEIs
25. juni 2015 SLIDE 29
• In progress
• Integration implemented and functioning
• Cooperation within the HE-sector to work on common
procedures for the student administration – a precondition
and essential requirement for simpler and cheaper systems
for the assessment administration
![Page 21: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/21.jpg)
National Norwegian
architecture for digital
summative assessments Bernt Smilden, Norwegian Business School
![Page 22: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/22.jpg)
Agenda
Drivers
Scope of architecture
Methods and inspirations to the approach
The architecture work
Further work
2. juni 2015 32
![Page 23: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/23.jpg)
Drivers
Technology drive
Political drive for mergers
Common legal regulations
Legal aspects of digital summative assessments
Requirement for equal treatment
Drive towards self service =>
digitalization of services
Bologna-process, ECTS
Shared learning objectives
National plans for learning
National exams
Collaboration on curriculum
Shared use of data across the sector
33
The drivers put heavy requirements BOTH to business and
IT!
![Page 24: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/24.jpg)
Methods and inspirations to the
approach
2. Juni 2015 35
Archimate® from Open Group
Archi Enterprise Architecture tool
TOGAF® Architecture Development
Method(ADM) from Open Group
![Page 25: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/25.jpg)
Inte
gra
tion
Standardisation
Coordination
Diversification
Unification
Replication
Source: ”Enterprise Architecture as Strategy: Creating a foundation for business
execution”
Major differences in operation between
business units/applications, integrated
into an extensive common business/IT
framework
Major differences in operation between
business units/applications, minimal
common framework
Minor differences in operation between
business units/applications on all levels,
minimal common framework
Minor differences in operation between
business units/applications on all level
integrated into an extensive business/IT
common framework
Standardisation and integration
![Page 26: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/26.jpg)
Inte
gra
tion
Standardisation
Coordination
Diversification
Unification
Replication
Source: ”Enterprise Architecture as Strategy: Creating a foundation for business
execution”
Business: Teacher governed formative
assessment types with few policies
IT: Individual and independent products
supporting each assessment type
Business: Each school is today an
autonomous unit within Norwegian
regulations
IT: Current Norwegian Common Student
Information System (FS)
Business: Future menu of summative
assessment types
IT: Modules for each assessment type
plugged into a unified exam framework
Business: Standardised future exam
processes and cross institution
cooperation
IT: One national digital exam solution
supporting the processes
Relevant examples and recommendations
![Page 27: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/27.jpg)
Detailed aspects of the architecture
Definition of specific processes
Information systems that support the processes
Information flows in and out of processes
Information accessed by the processes/activities
Information model
The application landscape in the neighbourhood of a digital exam solution
38
![Page 28: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/28.jpg)
Definition of processes Grouping of activities in specific processes characterised by:
Has a trigger
Uses input covered by the information model
Produces a result covered by the information model
Has a well-defined case being the information carrier for a process instance
Is significant enough to have a process manager with a follow-up responsibility
Example: Author test
39
![Page 29: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/29.jpg)
Process input and output
Describes the business’ process integration, putting requirements to the integration between applications and the information to flow through the integration
Describes the information formally
Is a significant input to and verification of the completeness of the information model
Example: Plan exam and Author test
41
![Page 30: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/30.jpg)
Application map
45
![Page 31: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/31.jpg)
Info
rmati
on a
rchit
ectu
re
47
![Page 32: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/32.jpg)
National project: Further work
Use the reference architecture: Verification & validation
Details for further architecture work:
Integrations
Additional specifications of collaboration between portals, LMS, question banks and exam
Requirement specification work
2. juni 2015 48
![Page 33: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/33.jpg)
Institutions: Candidate areas for further
work Analysis of as-is situation:
Administrative processes:
• Are the processes clearly defined?
• Are they harmonised within the sector?
Surrounding IT solutions:
• Are they in use?
• Do they actively support the processes
covered by the architecture?
• Are they ready for integration with a digital
exam solution?
Digital exam solution vendors:
• Does the solution support the processes in the
architecture?
• How complete is the information model?
• Which integrations do they support?
Activities to close gaps between as-
is and to-be architectures:
Administrative processes:
• Roll out standardised processes
IT solutions:
• Roll out available shared sector solutions
to support the processes covered by the
architecture
• Ensure that individual solutions
collaborate to reach the goals
2. juni 2015 49
The activities simplifies roll-out of a shared digital exam
solution
UNINETT supports and coordinates the work
![Page 34: Paper 39 – A digital assessment at Norwegian Universities](https://reader031.fdocuments.us/reader031/viewer/2022021815/586841c61a28ab40408be269/html5/thumbnails/34.jpg)
A mature HE-sector and a
mature market
https://www.uninett.no/digitaleksamen