ROI of Graderworks

by: Anthony Garland

Is Graderworks a good investment for an educational institution? To answer the question, we must contrast existing methods of grading SOLIDWORKS files with using Graderworks to assess SOLIDWORKS files. 

Graderworks adds value to an educational institution in two ways. 

  1. Saving time.
  2. Performing tasks which are not manually practical.

Assignment Types and Current Grading Methods

Graderworks saves time by automating the grading of many different kinds of assignments that are commonly given in engineering- graphics courses at universities or technical colleges. Since time = money, it is necessary to quantify the amount of time saved by using Graderworks for each assignment type. So, what are the types of assignments given in an engineering graphics course and how are they currently graded?

  1. Generate a SOLIDWORKS part file based on a 2D engineering drawing. This homework type is exactly the type of problem given on the SOLIDWORKS Associates Test (CSWA) which requires the student to make a 3D model based on the 2D orthogonal views of a part. For this type of assignment, the instructor has already generated the 3D model using SOLIDWORKS (.sldprt), made the 2D engineering drawing (.slddrw), saved the drawing in a format which can be easily given to the students (.pdf), and distributed the pdf(s) to the students. The students make the SOLIDWORKS part file and turn it into the teacher through email or a course management system. Grading the assignment requires the following steps.
    1. Opening each SOLIDWORKS file one by one. 
    2. Identify who made the model. Often, instructors require each student to put their name or student username in the file name of the part (anthonygarland_part5_homework6.sldprt), and then the instructor can quickly identify who made the part. 
    3. Assess the SOLIDWORKS part file. Two primary methods exist which an instructor might use to determine the 'correctness' of the model.
      1. Grade the mass properties. To speed up the grading process, SOLIDWORKS instructors often only grade the mass or volume of the part (similar to how SOLIDWORKS grades the CSWA), 
      2. Look at the final shape of the model to see if it looks correct. This method works well and allows for partial credit. However, this method is very subjective unless the instructor individually looks at each dimension of the part to verify it is correct. Besides, this method is very time-consuming. 
    4. Record the grade. The instructor must manually type or write down the grade for the student. 
    5. Closing the file. 
  2. Generate an engineering drawing (.slddrw) based on a 3D model. Learning how to communicate design information clearly is one of the primary goals of most engineering graphics courses, so teaching students how to generate 2D drawings based off of a 3D model is essential. Generating engineering drawings includes making both assembly and part file drawings. For this type of assignment, the students must turn in a .zip folder with both the parts and drawing files since the drawing files will not open without the part files. Grading these engineering drawings requires the following steps. 
    1. Unzip each folder one by one. 
    2. Identify who made the drawing. 
    3. Open the drawing file. 
    4. Assess the quality of the engineering drawing. 
    5. Record the grade. 
    6. Close the file. 
  3. Generate an assembly using SOLIDWORKS part files. Learning how to make assemblies is essential for SOLIDWORKS students. For this type of assignment, the instructor has already generated the SOLIDWORKS assembly and communicates to the students the correct relationships or mates between the parts using an engineering drawing or written text. Two types of assemblies exist 1.) Fully defined rigid assemblies, and 2.) Moving assemblies. The CSWA exam requires students to make rigid assemblies, but for machine design, these assemblies are not realistic since nearly all machines have moving parts. Students must turn in .zip files with the SOLIDWORKS part files and the SOLIDWORKS assembly file since the assembly will not open without the part files. Grading assemblies involves the following steps:
    1. Unzip each folder one by one. 
    2. Identify who made the assembly.
    3. Open the assembly. 
    4. Assess the assembly file. For rigid assemblies, the instructor can grade the center of mass. SOLIDWORKS uses this grading method to grade the CSWA assembly questions. For flexible assemblies, the instructor must assess if the assembly moves correctly and looks correct. The instructor evaluates by attempting to pull or drag on parts within the assembly to see if the assembly has the proper degrees of freedom. 
    5. Record the grade. 
    6. Close the file. 
  4. Generate non-SOLIDWORKS files using SOLIDWORKS. SOLIDWORKS is primarily a 3D modeling program, but it also offers many other tools which are helpful to engineers. Within an engineering graphics class, an instructor may expose students to these additional tools or add-ins within  SOLIDWORKS. These include PhotoView 360, animations, FEA, and more.  It is hard to describe the workflow required for each of these since each one is different. 
  5. Generate a SOLIDWORKS part or assembly based on a physical object or an original design. For this type of assignment, the instructor gives design specifications or a physical model and asks the students to generate the 3D models within SOLIDWORKS. No absolute 'correct' answer geometry exist since the assignment is partially open-ended. However, the instructor normally has several grading criteria that will be used to assess the quality of each student's work. For example, if the instructor gives the students a gearbox and asks them to produce a model within SOLIDWORKS, then likely the grading criteria will be that the gearbox functions properly (i.e. the gear mates all work and there is no interference between parts). Within an industrial engineering context, an instructor might ask the students to generate an ergonomic design for a drill handle. When assessing the assignment, the industrial-engineering instructor will carefully examine the drill handle while using standard ergonomic principles as his or her grading criteria. The grading workflow is similar to the 'Generate an assembly based on SOLIDWORKS part files' type of assignment. The only difference is that the assessment method requires the instructor to apply their unique grading criteria to the assembly. 

 

Graderworks: A New Method of Grading

Before contrasting Graderworks with manual grading of SOLIDWORKS files, a brief overview of the capabilities of Graderworks is needed. 

Graderworks is a software tool which iterates over 100s or 1000s of SOLIDWORKS files and automatically opens them, extracts mass property data and meta-data, and performs a geometric similarity comparison test on each file. The inputs to Graderworks are the location of the files to grade, the correct answer file (needed for the geometric comparison algorithm), the type assignment (SOLIDWORKS files or .zip files containing SOLIDWORKS files), and username identifiers to extract the username from the file or folder name. Graderworks outputs all of its data to a .CSV file. Within Microsoft Excel, this data can be easily converted into a grade automatically using macros (a recorded series of commands or a small computer program that runs within Excel). Also, Graderworks identifies possible plagiarism cases and exports a list of files which need further analysis to determine if the student plagiarized. 

By showing how instructors use Graderworks for each assignment type and contrasting it to the manual methods of grading, the advantages of Graderworks and its limitations (it cannot do everything) will become clear. 

  1. Generate a SOLIDWORKS part file based on a 2D engineering drawing. As a review, for this type of assignment, the instructor has already created the 3D model within SOLIDWORKS (.sldprt) and made the 2D engineering drawing (.slddrw).  The instructor saved the drawing in a format which can be easily given to the students (.pdf) and distributed the pdf(s) to the students. The students make the SOLIDWORKS part file and turn it into the teacher through email or a course management system. Grading the assignment with Graderworks requires the following steps.
    1. Configure Graderworks. Configuring Graderworks will require a maximum of 5 minutes. 
    2. Run Graderworks while working on other tasks. The rate at which Graderworks can analyze each file is primarily dependent on the speed of the hard disk and the amount of RAM available. However, since there is no action needed by the instructor while Graderworks is running, he or she can work on other tasks. (While running Graderworks on a laptop with an SSD and 6 GB of RAM, Graderworks analyzes about 1.3 files per second). 
    3. Run the grading macro. Once Graderworks is finished, the instructor opens the output.csv file in Microsoft Excel and runs the grading macro. This macro automatically objectively computes a grade for each student using the mass properties and geometric similarity comparison tool. Running the macro will take about 1 minute. **
    4. Check the analysis.csv file for plagiarism instances. 
  2. Generate an engineering drawing (.slddrw) based on a 3D model. As a review, for this type of assignment, the students must turn in a .zip folder with both the parts and drawing files since the drawing files will not open without the part files. Grading these engineering drawings requires the following steps. 
    1. Configure Graderworks. For this type of assignment, Graderworks will be configured to export a .png file of each drawing file. This option is under the advanced tab in the configuration window. Configuring Graderworks will require a maximum of 5 minutes. 
    2. Run Graderworks. 
    3. Open each .png file and assess the quality of the engineering drawing. Since Graderworks exported a .png of each drawing, there is no need to open the drawing file within SOLIDWORKS, but instead, the instructor can quickly look at each drawing on any image viewer program. Not opening SOLIDWORKS for each file, but instead using a lightweight image viewer to examine each drawing saves a significant amount of time while grading. 
    4. Record the grade. 
    5. Check the analysis.csv file for plagiarism instances. 
  3. Generate an assembly using SOLIDWORKS part files.  Students must turn in .zip files with the SOLIDWORKS part files and the SOLIDWORKS assembly file since the assembly will not open without the part files. Grading assemblies involves the following steps:
    1. Configure Graderworks. For this type of assignment, Graderworks must be configured to unzip each .zip file into a folder. The new folder will be named as the username of the student.
    2. Run Graderworks. 
    3. Compute the Grade for rigid assemblies. 
      1. For rigid assemblies, the instructor can open the output.csv file in Microsoft Excel, and run a grading macro to objectively compute a grade for each student. (this is how the CSWA assembly questions are graded).
      2. For flexible assemblies, the instructor must manually open the assembly files and determine a score, record the grade, and close the file. 
    4. Check the analysis.csv file for plagiarism instances. 
  4. Generate non-SOLIDWORKS files using SOLIDWORKS. Graderworks offers no significant advantages for grading non-SOLIDWORKS files, such as photo-realistic renderings, or animations of a machine working. Graderworks' advantages are programmatically unzipping .zip files to creating folders and identifying possible instances of plagiarism within the SOLIDWORKS files used to generate the renderings, animations, or other non-SOLIDWORKS files. 
  5. Generate a SOLIDWORKS part or assembly based on a physical object or an original design. For this type of assignment, no absolute 'correct' answer geometry exists since the homework is partially open-ended. The grading workflow involves: 
    1. Configure Graderworks.
    2. Run Graderworks.
    3. Compute a Grade. Programmatically computing a grade will likely not be possible. However, using Graderworks to unzip each student's .zip file into a folder can help save a significant amount of time. Also, depending on the specific grading criteria, saving the isometric screenshots of each part and assembly to a .png file could let the instructor avoid opening SOLIDWORKS for each student's assignment. Instead, the instructor could grade using the .png image which can be opened very quickly in a lightweight image viewer program. 
    4. Record the score
    5. Check the analysis.csv file for plagiarism instances. 

Time comparison

By automating many of the grading tasks, Graderworks frees up the instructor to work on other tasks. For tenure professors, this extra time can be used for research. For teachers at a technical college, this additional time can be used to prepare better for teaching or interaction with students. Graderworks adds value to any educational institution teaching SOLIDWORKS. However, the benefits do increase when trying to automate grading with large groups of students. 

For this time comparison between manual and Graderworks grading, a class size of 100 students will be used and the types of assignments that are given to the students throughout the semester are as follows. This distribution of assignments is based on an actual SOLIDWORKS class at a major public university

  1. Twenty quizzes which consist of students modeling a simple part or rigid assembly. 
  2. Ten homework assignments consisting of parts, assemblies, drawings, and non-SOLIDWORKS files. 
  3. Three major projects. Two of the projects are open-ended and do not have a 'correct answer'. 

 

Task  Grading Manually Grading with Graderworks
Quiz

20 seconds per file

6 minutes per assignment total
Homework 2 minutes per student's asignment

Half of the grading tasks can be automated.

1 minute per student's assignment

Projects 3  minutes per student

Automated unzipping and automation of half the grading tasks.

1.5 minutes per assignment 

 

Time with Manual Grading

Per student

(20 seconds)*(20 Quizzes) + (120 seconds)*(10 Homeworks) + (180 seconds * 3 Projects) = 2140 seconds = 35 minutes and 40 seconds

Total time for all students for one semester

(100 students * 2140 seconds/student) = 214,000 seconds =

2 days 11 hours 26 minutes 40 seconds

 

Time with Graderworks

Per student

(60 seconds)*(10 Homeworks) + (90 seconds * 3 Projects) = 870 seconds = 14 minutes 30 seconds

Total time for all students for one semester

(6 minutes)*(20 Quizzes) + (100 students * 870  seconds/student) = 94,200 =

1 day 2 hours 10 minutes

 

Converting time to money

The time difference between manual grading and using Graderworks is about 33 hours. Assuming that the instructor is paid at a rate of $25/hour the cost of manual grading for an extra 33 hours per semester is

(33 hours)*($25/hour) = $825

Graderworks currently costs $2.00 per student per semester. For each semester, the cost of Graderworks would be

(100 students)*($2.00 per student♦) = $200

Therefore when converting the 33 hours saved by using Graderworks and then subtracting the cost or Graderworks, the total savings for a semester is ($825- $200),

$625

Of course, when trying to quantify the value of Graderworks, the instructor should modify the numbers to their specific situation. Also, by slightly modifying assignments so that they have a clear, correct answer, nearly all of the grading could be automated which would increase the value of using Graderworks even more. 

Additional Value

Graderworks offers other benefits that have no direct comparison to manual grading. 

Objective Grading

Graderworks allows the instructor to assess the 'correctness' of files objectively with respect to the correct answer using the geometric similarity algorithm. For large classes, fair grading becomes more difficult since it is impossible to remember how many points were taken off for each type of mistake. Also, if a class has multiple instructors, then each person will likely grade slightly differently.  Graderworks eliminates this subjectiveness. 

Catch Plagiarism

Graderworks identifies potential instances of plagiarism. Students often try to share files instead of creating the parts themselves, get files from students who previously took the class, get files from online sources, and get files from a CD containing the completed assignments in the back of their textbook. Other courses at educational institutions use text-based plagiarism checkers, like TurnItIn. Graderworks is very similar and quickly identifies possible instances of plagiarism using the meta-data internal to the SOLIDWORKS files. Schools that use TurnItIn to analyze student's papers but do not use a plagiarism checker for SOLIDWORKS are being naive to think that students will not attempt to cheat in a CAD class. 

Flexibility

Graderworks does not lock the instructor into teaching SOLIDWORKS in a particular way, assigning specific assignments, or using a specific grading rubric. Graderworks exports data about each file to the output.csv file and then the instructor can decide how he or she wants to use the data to compute a grade. 

 

If you have questions or need help installing and running Graderworks, please contact us. 

Try Graderworks for free for one semester to see if it adds value to your institution. Download the trial version today!

 

** Garland Industries offers free help in customizing macros to meet your grading needs. 

♦ USA pricing. Contact Garland Industries for prices outside of the USA. 

*** The numbers shown in the image at the top of the page come from the following calculations. Manual grading requires 35 minutes and 40 seconds (calculations are shown above). Using Graderworks requires 94,200 seconds /100 students = 15 minutes and 40 seconds of grading per student.  The total cost of grading manually is 214,000 seconds * $25/hour = $1486.  When using Graderworks, the total cost of grading is 94,200 seconds *$25/hour+$200 (price of Graderworks) = $854.

 


Join our free mailing list!

Join our free mailing list!


Graderworks

  • Grade student's SOLIDWORKS files quickly
  • Catch Plagiarism
  • Export SOLIDWORKS files to another format in bulk
View Software

Recent Articles

Graderworks: SW 2017, Config files, Command line args

Graderworks 2.36 and higher is compatible with SOLIDWORKS 2017. Graderworks now


Graderworks: An Official SOLIDWORKS Solution Partner

Garland Industries is happy to announce that Graderworks is now an official SOL


Graderworks 2.35

Several more improvements have been made to Graderworks in version 2.35.


Graderworks 2.32 Release

Graderworks 2.32 represents continued incremental improvement in the overall qua


ROI of Graderworks

Graderworks saves educational institutions $625 per semester for a Solidworks en


Design Automation Example

A design automation example is given showing topology optimization coupled with


Graderworks 2.31

Graderworks 2.31 includes the following updates. 1. When analyzing .stl files th


Graderworks 2.3 Update

Update on Graderworks 2.3 Several major bugs were fixed.


Graderworks 2.3 Release

Graderworks 2.3 allows you to compare the geometric similarity of .stl files wit


Turnitin vs. Graderworks

Graderworks is the Turnitin of 3D models. Not using a plagiarism checker in a So


Getting Started with Graderworks

This tutorial shows how to get started analyzing and grading Solidworks part fil


The Making of Graderworks Video

As an engineering entrepreneur, marketing is not my strength. To make high-quali


Organizing hundreds of .stl files for 3D printing

How would you 3D print hundreds of .stl files in the shortest time possible usin


What is unique about 3D printing?

3D Printing offers many new exciting possibilities, but why is it unique? Three


How 3D printing works

3D Printing requires three steps. 1. Getting a 3D model. 2. Slicing the model to


Preprocessing files for Graderworks: Converting .rar to .zip

Solidworks students submit .rar files instead of .zip. Graderworks requires .zip


Windows Smart Screen and Code Signing

A short tutorial on how to sign a Windows installer and what the smart screen fi


What should we 3D print?

3D Printing opens the possibility of mass customization of consumer products. Ma


How did Graderworks Evolve?

How would you grade 1000s of Solidworks file? This article is the story of how I


Software, dying on the journey from academic project to commercial tool. 

Does software make it from academic research to useful commercial tools? Ignoran