How did Graderworks Evolve?

by: Anthony Garland

How did I come up with Graderworks? How did it evolve to its current state? I often get asked these two questions, so here are the answers. 

How did I come up with Graderworks?

I started teaching SOLIDWORKS as a graduate teaching assistant at Clemson University 2.5 years ago when I started my Ph.D. The SOLIDWORKS curriculum at Clemson has pros and cons (how you should teach engineering graphics in a university setting is a whole topic in of itself). A strength of the Clemson curriculum is preparing students for taking the Certified SOLIDWORKS Associate (CSWA) exam by giving 'quizzes'  each class period (we call them "in class assignments" but they are quizzes). The quizzes help the students prepare for the CSWA by giving them practice modeling in SOLIDWORKS while under a time limit. The question is if you have 300-500 students taking the class each semester, and if you assign two quizzes each week, then how will you grade 600 or 1000 files every week?

Method 1: Live Grading

When I first started teaching SOLIDWORKS at the end of each class period, I would go around and look at each student's computer to see if they drew the part correctly. The class had about 40 students, and most students finished within the last few minutes. We asked the students to show us the mass properties window which let us quickly see if their part had the correct mass. The assumption was that if the mass is right, then the part must be correct. This assumption is the same assumption that the CSWA makes while grading. The problem was that in a class of 40 students, there was no way I could walk around and check each person's computer in the last 3 minutes of class while at the same time trying to help some student figure out what they were doing wrong. Students were getting frustrated, and so were the teachers of the class. I'll call this the "live grading" method. We need a solution, and we came up with two. 

Method 2: Self-reporting Grading

The second method for grading these quizzes requires students to self-report their mass (or center of mass for an assembly) to the online course management system. The students would insert a numeric value to the web page, and it would tell them if it was within the acceptable range. We also require students to upload the actual SOLIDWORKS part file (.sldpart) to the course management system, but we did not check these parts. I'll call this the "Self-reporting" method. This system worked much better than "live grading", but it still had some flaws. The obvious flaw is that my friend Bob could say "Hey, Anthony, the correct mass is 53.4 grams." Then I could go to the course management website and insert 53.4 grams and upload my SOLIDWORKS file that had a mass of 20 grams because I couldn't figure out how to model the part, and I will still get a grade of 100%  since only the numeric value that I self-reported was being graded.  Obviously, cheating or plagiarism was easy, and we didn't have a good way to check for this. 

While we start the new "self-reporting" method, I began working on seeing if I could use the SOLIDWORKS API to extract the mass properties automatically. In fact, at the end of my first-semester teaching, I had even suggested to my superiors that they look into this idea. Using the SOLIDWORKS API is simple, but only if you know what you are doing! (Isn't that true of most things in life.) Since I've accumulated a good bit of experience using the .NET framework with C#, using the SOLIDWORKS API was pretty straight forward. At first, I wrote a simple hard-coded command line program to loop over all the SOLIDWORKS parts .sldprt in a folder from all the students taking SOLIDWORKS at Clemson (about 300 at the time) and extract the properties to a .csv file. The results were immediately impressive! 

  1. First, I now had all the mass properties for every file. We could use this for grading. It took my superiors a while to decide that we should use this information for grading, but we eventually used the density to determine if they assigned the right material, and the volume to see if they had the correct geometry. The rubric became 25 points for assigning the proper material and 75 points for the correct geometry. 
  2. Second, I sorted the files by the 'creation date' extracted from the SOLIDWORKS file. The creation date internal to the SOLIDWORKS file cannot be changed and is not the same thing as the Microsoft Windows creation date. SOLIDWORKS records when the file was made down to the second. When I sorted the files, I noticed several interesting things. 
    1. Some students 'made' their files a 'long time ago'. How is it that a student could turn in a file created nine months ago, but this is their first time taking the SOLIDWORKS class? And it's their first time using SOLIDWORKS?
    2. Some students had the same 'creation date', meaning that creation date had the same year, month, day, hour, minute, and second. How did that happen? Did they actually 'create' the file at the same time?

Method 3: Graderworks

Things change slowly at a public university, so it took a while before we started using Graderworks and excel formulas to compute a grade for each student. We continued to use method 2 for a semester. When a new course coordinator took over, she continued the grading process that we had been using in the past (method 2). However, about 1/3 of the way through the semester, she accidently forgot to open the numeric mass submission form on the webpage course management system for one of the quizzes. The result was that students did not report their mass, but they uploaded their SOLIDWORKS part files. How to grade these quizzes? Naturally, I suggested we use Graderworks to extract the mass properties and use the mass properties for grading. This new method is what we did, and we have been using it ever since then. 

extracted properties from the Solidworks files

Those two fundamental ideas are the core of Graderworks. 1.) Exporting basic properties and 2.) Catching plagiarism using the meta-data extracted from the files. Since then I've improved Graderworks in many ways so that it analyzes the files in much more sophisticated ways both for grading and catching plagiarism.

Method 4: Graderworks with partial credit

A critique of method 3 is that the all-or-nothing method of grading does not reflect a student's understanding or ability. If a student draws a cylinder with the right material, he will get a grade of 25%. If a student draws a part 98% correct and has the right material, the mass of the part will be incorrect, and he will get a grade of 25% as well. Unequal performance should not receive equal grades. At Clemson, this was a major issue since often students would get a low grade on a quiz for a tiny mistake. Often students couldn't even find their mistake which caused their mass to be slightly incorrect. So, we needed a way to assign partial credit.

My office mate in the mechanical engineering department at Clemson was researching using geometric similarity of 3D models to predict the assembly instructions of machines. I implemented into Graderworks a heavily modified version of one of the geometric similarity algorithms that he used in his research. This new algorithm allows Graderworks to judge the geometric similarity between the correct answer model and the student's model which then allows a teacher to assign partial credit. A small mistake that only slightly modifies the geometry will now only cost a student a few points instead of getting a grade of 0% or 25%. A simple example of this technique is found on the examples page, and a more comprehensive step-by-step tutorial is located in the blog article "Getting Started with Graderworks. This geometric similarity comparison algorithm was a game changer since now Graderworks can intelligently grade files similar to how a human would. In fact, the whole system could be automated to remove much of the work of the teacher. 

Evolving into a Commercial Product

I decided to turn Graderworks into a commercial software tool when I realized that the tool could be very helpful to other universities and community colleges. So, how should it evolve in the future? And what are the guiding principles? The goal of Graderworks is to minimize the time spent on limited-value tasks. Grading is necessary, but spending time interacting with students is much more valuable.

The goal of Graderworks is to limit time on non-value added taks

The Future

In the future as Graderworks changes with ever advanced grading algorithms, it could potentially eliminate all grading tasks for the teacher.  With all the SOLIDWORKS educational material available online and published in books, could we remove the teacher entirely? The answer depends on what we believe is the role of a teacher. I firmly believe that the goal of the teacher is to facilitate the learning process as much as is practically possible. To the extent that Graderworks frees up teachers so that they can work directly with students to facility the student's learning, is the degree to which Graderworks is a valuable, helpful tool for educators. 

 


Join our free mailing list!

Join our free mailing list!


Graderworks

  • Grade student's SOLIDWORKS files quickly
  • Catch Plagiarism
  • Export SOLIDWORKS files to another format in bulk
View Software

Recent Articles

Graderworks: SW 2017, Config files, Command line args

Graderworks 2.36 and higher is compatible with SOLIDWORKS 2017. Graderworks now


Graderworks: An Official SOLIDWORKS Solution Partner

Garland Industries is happy to announce that Graderworks is now an official SOL


Graderworks 2.35

Several more improvements have been made to Graderworks in version 2.35.


Graderworks 2.32 Release

Graderworks 2.32 represents continued incremental improvement in the overall qua


ROI of Graderworks

Graderworks saves educational institutions $625 per semester for a Solidworks en


Design Automation Example

A design automation example is given showing topology optimization coupled with


Graderworks 2.31

Graderworks 2.31 includes the following updates. 1. When analyzing .stl files th


Graderworks 2.3 Update

Update on Graderworks 2.3 Several major bugs were fixed.


Graderworks 2.3 Release

Graderworks 2.3 allows you to compare the geometric similarity of .stl files wit


Turnitin vs. Graderworks

Graderworks is the Turnitin of 3D models. Not using a plagiarism checker in a So


Getting Started with Graderworks

This tutorial shows how to get started analyzing and grading Solidworks part fil


The Making of Graderworks Video

As an engineering entrepreneur, marketing is not my strength. To make high-quali


Organizing hundreds of .stl files for 3D printing

How would you 3D print hundreds of .stl files in the shortest time possible usin


What is unique about 3D printing?

3D Printing offers many new exciting possibilities, but why is it unique? Three


How 3D printing works

3D Printing requires three steps. 1. Getting a 3D model. 2. Slicing the model to


Preprocessing files for Graderworks: Converting .rar to .zip

Solidworks students submit .rar files instead of .zip. Graderworks requires .zip


Windows Smart Screen and Code Signing

A short tutorial on how to sign a Windows installer and what the smart screen fi


What should we 3D print?

3D Printing opens the possibility of mass customization of consumer products. Ma


How did Graderworks Evolve?

How would you grade 1000s of Solidworks file? This article is the story of how I


Software, dying on the journey from academic project to commercial tool. 

Does software make it from academic research to useful commercial tools? Ignoran