Ideas for improving the QGIS certification programme
The QGIS Certificate Program is designed to promote both community involvement in the QGIS project and quality education for QGIS software. As such the application process requires applicants to both detail their contributions to the QGIS project and make their training materials available for review.
You may be wondering about our approach to certification. We looked at a number of approaches where we could provide a more rigorous and systematic way of certifying software users. These approaches can be split in two general approaches:
- Computer based testing: Favoured by industry, in this approach the project owner would create a large pool of questions and then provide a test environment where a random subset of questions are allocated to the person being tested and they would carry out an online examination. There are a few issues with this approach including the need for a way to verify that the correct user actually undertook the exam and not a proxy, the need for a large amount of project input to create and maintain the test questions, and the general trend for examinees to go online and share the questions the were asked for future examinees. There is also the issue that sometimes complex (especially graphical) software is hard to examine in an environment where text based answers are expected.
- Portfolio based testing: Favoured by academia, this approach requires attendees to submit mini-projects or other projects such as maps and reports to demonstrate their competency. This has a downside that each evaluation is very personal and subjective.
Neither of these systems is really suitable for our needs so we have opted for a third choice: In this approach we create a "web of trust". In this approach we trust the organisations that provide training to properly issue certificates for their course attendees. We spend our time verifying the certifying organisation and then trust them to 'do the right thing' after that.
In the future we might want to change that, but currently it has no priority.
Reviewers: Kurt Menke, Hans van der Kwast
Platform: Tim Sutton, Anita Hapsari
Billing: Andreas Neumann
An advisory board consisting of Lene Fischer and Erik Meerburg has been proposed given their experience in GIS education and their knowledge of community building.
Training materials should be provided for review of their quality. Training materials should be using at least the latest QGIS LTR version. The LTR version will be printed on the certificate. Re-assessment needs of the training materials should be done after the release of an LTR version. The review will be done by Kurt Menke and Hans van der Kwast.
Any other criteria?
How do we deal with language?
Distribution of tasks for review?
Contributions to the QGIS project include activities such as:
- Development/commits to QGIS
- Contributions to the QGIS documentation and training materials
- Translation of QGIS materials
- Authoring plugins
- Filing bug reports
- Financially supporting new feature development
- Financial contributions to and sponsorship of QGIS
- Involvement in your local QGIS User Group
- Publication of open QGIS course ware
- Authoring QGIS books
The certification platform in Projecta is designed to provide an easy path for projects to be able to issue certification certificates. The system allows a project to approve one or more certification providers. These providers can then set up training centers and course conveners. Course conveners prepare a course and then attendees are assigned to that course. Once the course is completed, certificates get issued by the course convener in digital form. There is the opportunity to charge a fee for each certificate issued. This mechanism can be used to support the parent project.
The following diagram provides a generalised overview of the certification process from the point of view of a certifying organisation.
Organisations that want to apply for the certification programme can submit an application form through the QGIS certification web page.
In the review phase the criteria mentioned above will be checked by the reviewers. Here's the workflow of the review phase:
- The reviewers receive a notification of new applicants. Not working well
- After reviewing the application form, the reviewers request the training materials for review. The applicant needs to indicate if and what assessment takes place. For the moment this will not be used as a criteria for certification, but might be used in the future. Proprietary materials will be treated confidential. The applicant is also asked to provide information on the contribution to the QGIS project. The latter will be confirmed with the QGIS User Group of the applicant's country. If that doesn't exist this will be judged by the PSC.
- The reviewer informs the applicant about acception/rejection.
In the case of a dispute about the decision the PSC will communicate with the applicant.
Once the organisation has passed approval, the organisation owner and managers will be able to proceed with setting up courses, conveners and so on using the management interface provided on Projecta. This is explained in the next section.
We need to decide how to distribute the review between the reviewers, maybe set criteria for that such as conflict of interest, language, course themes, etc. Suggestion to add Lene Fischer and Erik Meerburg as advisors
Notes from Kurt
There is a feature I have requested that I believe would greatly help manage the applications. I would like to have space next to each for the date of contact and the status of the application (awaiting response | awaiting materials | in review). It becomes difficult to track who has been contacted, who has responded but is preparing their application etc. Many never respond or take a long time to respond. Many also need time (months) to update materials or prepare their application. With this feature it would be much easier to track and determine who needs initial contact or follow up.I requested something like this in October 2018 https://github.com/kartoza/prj.app/issues/980
Once the organisation is approved, the applicant will receive a confirmation email and the organisation can be managed in the Projecta platform. The Projecta platform is descibed in detail here. Here we'll present the workflow.
The about page with Projecta info should be more easily found.
- Applicant logs in at Projecta
- Go to Certification -> Organisations
- CLick Sign Up to sign up for certification
- On the registration page you can fill in all the key details about your organisation including its, address (physical and online), contact person email, logo and so on. It is important to include a logo if you have one as that will be used on the certificates issued by the system. We recommend that you use a logo with a square aspect ratio for this. We need to implement that the applicant will have to give a consent for visibility of contact information (in line with GDPR).
- After that you can see your organisation page, which gives a quick dashboard overview of your organisation, credits, training centers and attendees. The map will be zoomed in to the region where your organisation operates. There are then four main things that can be managed:
- Courses
- Training Centers
- Course Types
- Course conveners
Typically you will first create Training Centers, Course conveners, Course types and then create courses.
A training center is a venue where you regularly conduct courses. Each organisation may have one or more training centers. If you are doing onsite or 'roaming' training, I suggest to either make a training center per location where you offer training, or create a single location (e.g. at your offices) that you can use as a proxy for other venues.
In the future we will provide a map with pins showing where upcoming training courses will be provided for all Certification organisations.
A course convener is either the trainer or the person responsible for the course. The certificate will be issued with the name of the course convener on it. Be sure to upload a digital signature for each course convener if you do not want to manually print and sign the certificates.
Course types describe the different content offering you offer. You can include how many hours of instruction that are included in the course. If you do this, this information will be included on the certificate when it is generated. Probably you will define your course types around topics/themes (e.g. working with databases) or around skill levels (beginner, intermediate etc.). You can also include a link to more course information.
A course is run at a training center by a course convener for a particular course. It requires to:
- Define course type (see above)
- Provide course convener (only 1 can be chosen, that's the convener who's signature will be added to the certificate).
- Select the training center for the course.
- Select start and end date of the course
- You can optionally add a background template which will be added to the general certificate template when it's generated. Note that the template you upload should be formatted to A4, not US Letter.
- You can preview the certificate
- Submit the course info.
After the course has been added to the dashboard of the organisation, the participants can be added by clicking the view icon next to the course. Then you can:
- Add course attendees one by one and assign them to the course
- Import attendees from a CSV file
- Issue certificates Now only one-by-one, would be nice to be able to select all and remove the ones that didn't meet the requirement
- Delete course attendee
- Download all certificates from the course
- Re-generate all pdf's from the course
- E-mail all participants the link to their certificate
The process of registering an attendee and issuing the certificate is separated so you need only issue certificate to those that pass your course. Each attendee will receive an email with a link to a downloadable copy of the certificate. The certificate includes a unique number which is allocated to each certificate and a unique link that they can provide (for example in their CV) to verify that they have indeed completed the course. This mechanism should help to prevent fake certificates. Course conveners should upload a digital signature to their profile. This is embedded into the certificate together with the digital signature of the project leader (who needs to set this in the project profile options page).
We'll add a link to the about page, so the employer knows what the QGIS certification means.
We need to hide the view button from the course page in order to hide the participant lists of courses (GDPR compliance).
We are planning to introduce an e-commerce system soon. When that is implemented, you will be able to purchase credits which can be used to then pay for the certificates you issue. Currently we work on a somewhat manual system - when you are going to be running a course, ask one of the project administrators to load credits for your organisation.
_Currently we charge 20 euros per certificate which will be 100% used for QGIS. We're discussing to use part of the income to sustain the certification platform. At this moment we'll not deviate from a flat rate to keep it simple. In the future we might want to have multiple rates for different regions."
Currently there's no way to check credits and issued certificates. If the applicant issued a certificate, one credit will be removed. However, when this shouldn't be done if:
-There's an error in the name and a new certificate needs to be made for the attendee
-Courses where attendees need to receive the certificate on the last day of the course. In that case the certificates are generated and printed in advance of the course. If attendees drop out or there are no-shows, these should be removed without lowering the credits.
If you have any problems using the certification platform please file an issue at https://github.com/kartoza/projecta/issues.
We need to prioritize issues. We might make a budget available for the sustainability of the platform, features and bug fixing. In the future the QGIS certification platform will run on its own instance.
We have concluded that QGIS User Groups are not allowed to issue certificates for their trainings.
- Feed for training centers, previous and upcoming courses etc. (https://github.com/kartoza/prj.app/pull/1162)
- Map on QGIS.org for training centers and upcoming training centers
- Onramping instructions for new orgs (already added Certification -> Learn about certification)
- Certifying org expiry every 3 years
- Post training anonymous survey to establish ratings for orgs