Graderworks Usermanual


Overview

Graderworks is a grading support software for SOLIDWORKS educators. Graderworks can analyze students' files, extract information, and perform routine grading tasks, which will free you up to perform higher-level critiques of your students' files.

This user manual is for Graderworks version 4.00 and higher.

The user manual for older versions of Graderworks is here

Installation

Requirements

You must have SOLIDWORKS installed on your computer for Graderworks to run.

We recommend that your computer have a solid-state hard drive. Since Graderworks opens hundreds of SOLIDWORKS files, it needs to be able to read from the disk quickly, which is why an SSD is helpful. Imagine how long it takes your computer to fully open a SOLIDWORKS file, then multiply this by the number of files you need to grade, and you will get approximately how long it will take for Graderworks to analyze your files. Also, a fast processor and at least 8GB of memory are helpful.

How to Install

  1. Download the installer from the downloads page.
  2. For each version of SOLIDWORKS, there is a different version of Graderworks. Be sure you select the correct version when you download the software.
  3. Run the installer to install Graderworks fully. (If you already have Graderworks installed, then you will be prompted to uninstall your previous version).

Quick Start

Basic Data Collection

  1. Put several SOLIDWORKS part files in a single folder on your computer.
  2. Run Graderworks.
    1. Click on the 'Assignment Manager' and create a new assignment.
    2. Set the root folder (the folder that contains the SOLIDWORKS files you want to grade).
    3. At the bottom in the "Analysis" section, click "Start."
    4. When Graderworks is finished, click on the hyperlink to the excel file with your data.

Data Collection with Automated Grading

  1. Put several SOLIDWORKS part files in a single folder on your computer.
  2. Run Graderworks.
    1. Click on the 'Assignment Manager' and create a new assignment.
    2. Set the root folder (the folder that contains the SOLIDWORKS files you want to grade).
    3. Click 'Data Collection Manager' to configure what types of files to analyze and how to extract the student's Username.
    4. Click 'Grading and Plagiarism Manager'
      • In the first tab, create at least one rubric item.
      • Use the second tab to configure the late penalty
    5. At the bottom in the "Analysis" section, click "Start."
    6. When Graderworks is finished, click on the hyperlink to the excel file with your data.

Graderworks Overview

When you first run Graderworks, the main page will show you some basic information about the software. In the message area, you will see information about your License, Graderworks version, and the version of SOLIDWORKS that is compatible with Graderworks.

Assignment Manager

To run Graderworks, you must select an assignment using the assignment manager. Assignments are the highest level of data organization within Graderworks. Configurations, rubric items, and student's data all belong to a single assignment.

With the assignment manager, you can create a new assignment or select an existing assignment. Also, you can rename assignments, show metrics about the assignment, delete assignments, delete an assignment's data (but not any other configurations), and copy an assignment.

Root Folder

In the "Root Folder" group, you can see the current root folder and also browse for a different root folder. The root folder is the folder containing the SOLIDWORKS files, folders with SOLIDWORKS files, or .zip files containing SOLIDWORKS files which you wish to analyze. The default root folder is the desktop. If your student's files are not on the desktop, then you must change the root folder. Using the desktop as the root folder will result in a warning from Graderworks.

Data Collection Manager

The data collection manager allows you to configure how Graderworks will collect data.

1. Specify Submission Type

Graderworks can grade various kinds of assignments.

  1. SOLIDWORKS files in the root folder. A student can submit more than one file.
  2. Folders in the root folder. The folder name will be analyzed to extract the Username. All the SOLIDWORKS files in the folder should belong to a single student.
  3. .zip files in the root folder. The .zip file name will be analyzed to extract the Username. The .zip file will be unzipped into a folder with the name of the extracted Username. All the SOLIDWORKS files in the .zip file belong to that student.
  4. Individual SOLIDWORKS files specified on the individual files window (The individual files window is not visible until you select 'individual files'). These files are not necessarily in the root folder. The Graderworks output files will still be in the root folder.

2. Types of SOLIDWORKS Files

Graderworks lets you select what types of files you would like to analyze. Unchecking any of the boxes to exclude a particular type of SOLIDWORKS file.

3. Username Identification

In the "Username Identification" group, you must specify to Graderworks how to identify each student. Graderworks expects each student submission to have a particular format. Most schools use some learning management system (LMS) like Canvas, Blackboard, Moodle, Schoology, D2L, or some other system. These systems allow you to batch download your student submissions to your computer. During this download, the LMS will rename your students' submissions with some sort of identifier in the file name. The 'Username Identification' within Graderworks is designed to help extract the username identifier from the filename. Within Graderworks, you must specify the beginning character before the LMS added identifier, and you must specify the ending character after the LMS added identifier.

If you do not use an LMS with batch download and rename capabilities, then you could tell your students to use a particular naming convention in their file names (like appending their Username and then an (-) dash).

The advanced section of the user manual will discuss how to map the extracted Username to a student's email address, first name, and last name.

Method 1. Beginning and Ending Username Identifier

In the table below, the Start Identifier is blank, so Graderworks will start at the beginning of the file. The End Identifier is set to an underscore (_). The last column in the table shows the name that Graderworks will associate with each file.

Student's filename Student's school username File renamed by LMS Graderworks extracted username
box_with_hole.sldprt bob5 bob5_box_with_hole.sldprt bob5
box_with_hole.sldprt joe789 job789_box_withhole.sldprt joe789

Once you have selected your root folder, type of assignment, type of SOLIDWORKS files, and set the username identification, then you can preview how Graderworks will extract the usernames by clicking "preview extraction." Graderworks will launch a new window that shows you what files or folders it found in the root folder and how it plans to extract the usernames.

Method 2. Regular Expressions

Extracting the Username from a file name may be more complicated than specifying the beginning and ending character of the student's Username. Regular expressions provide a convenient method to extract the Username from any file. Regular expressions are a general syntax for extracting a sub-string from a string. You can learn about the required syntax here from Microsoft. If you need help, contact help@garlandindustriesllc.com.

4. Preview Username Extraction

Click 'Preview Extraction' to view how Graderworks will extract the Username from files in the root folder.

When you are finished configuring how Graderworks should collect data, then close the window or click 'Save Settings'.


Rubric and Grading Manager

The rubric and grading manager allows you to configure the automated grading of student's SOLIDWORKS files.

Creating a Rubric

The Graderworks rubric allows you to compare student's models to the correct answer models. Each rubric is specific to an assignment, and each rubric has one or more rubric items. Within each rubric item, there are one or more grading items.

Graderworks can automate grading of SOLIDWORKS parts and rigid assembly files.

Within the Rubric and Grading Manager, click on the "Rubric" tab. The rubric tab allows you to make, create, edit, and delete rubric items within your rubric. Rubrics are specific to an assignment, so if you change the assignment, then your rubric will save, and any rubric information for the new assignment will be loaded.

Creating Rubric Items

In the rubric tab, click on "Create Rubric Item." A new window will open. Each rubric item has many options for how to apply the rubric item.

  1. Basics
    1. Name. You should give each rubric item a meaningful name.
    2. Percent of total assignment score. This field tells Graderworks how much each rubric item counts toward the overall assignment score. If you have just one rubric item for your assignment, then this field should be 100. If you have more than one rubric item, then weight each one so that the total weights add up to 100. For example, rubric item 1 might count for 35% of the assignment grade, and rubric item 2 might count for 65% of the assignment grade.
    3. File path to the correct answer. You must tell Graderworks what the correct answer model. Browse for the model on your computer to select the file path to the correct answer model. Or, you can type or paste in the file path to the correct answer model.
  2. How to Grade.
    1. Graderworks can grade the following items.
      1. Volume. Graderworks will compare the student's volume to the solution's volume.
      2. Material. Graderworks will analyze the student's material (by checking the density) to the solution's material.
      3. Composite Shape Score. Graderworks will compare the shape of the student's file to the solution's shape. This check is accomplished by aggregating several different metrics about each file. The composite shape score only considers the actual shapes of the parts and not how they were drawn. Part of the shape score is based on a stochastic process with measurable variability. The last three lines of the "raw data" sheet in the output file quantify this variability (generally less than 3%). Since the shape score is based on a stochastic process, the composite shape score should be treated as the center value of a range of potential shape scores. When filling out the rubric for each student and computing file grades, Graderworks considers this variability when assigning a composite shape score of 100%. For example, the raw shape score is 98.5%, and the estimated error is 2%, then the file is within the quantifiable range of acceptable shape scores. If a file's shape score is near 100%, then Graderworks will perform additional checks to determine if the file is 100% the same shape as the solution or just really close to the same shape as the solution's shape. These additional checks are very accurate and precise. If the composite shape score is 100%, then you are guaranteed that the solution shape and the student's model's shape are the same.
      4. Center of mass. Graderworks will compare the center of mass of the student's file to the solution's center of mass. This grading item helps grade the orientation and origin of a model or a rigid assembly (like on the CSWA assembly questions).
      5. Fully defined sketches. Graderworks will check all sketches in the student's file and identify any that are not fully defined. Graderworks will list the underdefined sketches in the output file.
    2. For each grading item, you can select the weighted that the grading item will contribute toward the rubric item. The percentages should sum to 100.

Once you are finished making your rubric item, then click "save." You can add more than one rubric item. Graderworks will automatically determine which of the student's models match which rubric item. So if students should submit five files for an assignment, and if you made five rubric items, Graderworks will make sure it applies the correct rubric item to each of the student's files.

Late Penalty

Graderworks can check each file that was graded to see if the last modified date-time is after a predefined date-time, which is the assignment due date. If a single file is modified after the due date, then Graderworks can take off a percentage of the student's overall grade. You must select the calendar day and time that the assignment is due. To use the late work check, you must also specify the percentage of points to take off.


Plagiarism Checking

Graderworks collects meta-data about the student's files, including the creation date, last saved date, and saved by name. These properties are internal to the SOLIDWORKS file and are NOT the windows os equivalent. Students can't modify these properties in the SOLIDWORKS file (if you find a way, then let us know).

Graderworks uses this meta-data to check for student plagiarism. Graderworks only identifies files that meet certain criteria that are suspicious or unusual; however, what constitutes plagiarism is dependent on how you set up the assignment and what exactly you told your students. Graderworks is only a tool to help you. You are the ultimate judge of plagiarism. Not Graderworks.

Advice on interpreting plagiarism checks

A few helpful pieces of advice can steer you in the correct direction while checking for plagiarism.

  1. If you suspect plagiarism, then always check the feature meta-data. If two students turn in files turn in separate files but all of the features are created at the same time (the date, hour, minute, second) by the same Username, then you can probably conclude that the two files are actually the same file and that one student obtained the file from the other student. Graderworks exports the feature data in the "raw data" sheet of the output Excel workbook.
  2. Check all of the features. Often students download a part from Grabcad or some other source, then modify it and turn it in. The first few features will likely have a much different creation date than last features.
  3. Don't assume that if two students have the same "saved by" name that they shared files. They could have the same Username on their computer, or they could have used the same computer. Again, the key is if both files have all or some of the features created at the same time. It is improbable that many of the features would be created at the same time even for students who use the same 'saved by" name.
  4. Showing the feature information (including the created by, date created, and date modified) of two files from two different students side by side is an excellent way to show a non-engineer why you suspect a student plagiarised a file. Some employees at GI LLC, have extensive experience in catching and prosecuting plagiarism in a SOLIDWORKS class, and we recommend this side by side comparison. Often in academic institutions, plagiarism accusations must be presented before some review board that is made up of non-engineers who do not understand what SOLIDWORKS is or how it is used. The review board will often understand your case if you show the feature information side by side (and the creation date and created by is the same for many of the features), tell the review board that SOLIDWORKS has "track changes" perpetually on (similar to Microsoft Word's track changes), and explain that two students could not have created by multiple features at exactly the same date, hour, minute, and second.
  5. If a student is retaking a SOLIDWORKS class, then his or her file creation dates might be much older than everyone else's creation dates in your class.
  6. Checking engineering drawings for plagiarism is hard if you only check the .slddrw file. The .slddrw (drawing file) does not contain much information that helps check for plagiarism.
  7. Graderworks cannot check for some situations. For example, if a student pays someone else to do their homework for them. Although if you suspect this situation, then a pop quiz during class will likely reveal who knows how to model in SOLIDWORKS.
  8. SOLIDWORKS parts from the Toolbox will be very old. If you require your students to use the Toolbox while making their assemblies, then you will notice that the toolbox part files were created by someone a very long time ago. If you told your students to use the Toolbox, then you can ignore these files.
  9. Sometimes a student may steal a SOLIDWORKS file from another student without their knowledge. The only way you could find this out is by talking with the students.

Graderworks can perform up to five different types of checks, which can help you identify plagiarism. Check the boxes next to the kinds of checks you would like Graderworks to perform.

More than one 'saved by' name for a single student

If a single student submits several files, then you would expect that the last saved by name would be the same for all the files. If they are not the same, then you might find this situation suspicious. Graderworks can identify these situations for you.

What if a student borrows his friend's computer to complete part of the assignment. The last saved by name could change for a single student even though the student did all of his own work. In this case, you should check the feature information. Even if they used the same computer, then the feature creation times will be much different if they both did their own work on the same computer.

The 'saved by' name is the same for different students

If two students turn in files and the last saved by name is "bob" for both students, then you might be suspicious. Again, you should check the feature information to obtain a better understanding of how the files were created.

The 'creation date' is the same, but different students made the files

If two students turn in files that were made on the same day, hour, minute, and second then you might suspect that one file is just a copy of the other file. You should check the feature information.

We have found this check to be very helpful and can identify most plagiarism instances when a student shares a file with another student.

The 'creation date' is less than two standard deviations from the average

If a file was created a long time ago, then you might suspect that the student obtained the file from another source (like Grabcad). This check tries to guess a reasonable limit on the create date (average minus two standard deviations). The average and standard deviations are based on all files turned in by all students for the selected assignment.

If your students are modeling an assembly that requires parts from a source that you give them or from the Toolbox, then this plagiarism check might not be the best. For example, some toolbox parts were created in 1998. If the student turns in a part file from 1998, then the average creation date of all the files in an assignment will be much earlier, and the standard deviation will be very large. In these instances, a student could have obtained a file from someone last semester and still be within two standard deviations of the average. Because of this potential issue, you might find the next check more helpful.

The 'creation date' is less than the date specified below

Similar to the previous check, this plagiarism check seeks to identify files made outside the expected time frame. Since most files obtained from an unauthorized source were not created during the current semester, this is an excellent check to find these files from unauthorized sources. For this check to work, you must specify an oldest allowed creation date. Any file created before this specific date will be identified.

Mail Merge Filled out Rubrics

Graderworks can automatically set up a mail-merge of filled out rubrics for you to send to your students via email or individual print and return to your students. To enable this feature, check the box in the Display Results group. The mail merged document is automatically generated. When you open the document, Microsoft Word will run an SQL command to make sure it has the correct data. Click 'Yes' when prompted at startup.

For general information about using Microsoft Mail merge, read this documentation from Microsoft

You Must have Microsoft Outlook configured on your computer for the mail merge to work.

With the mail merge document open, follow these steps.

  1. Click on the 'Mailings' tab in the ribbon.
  2. In the 'Preview Results' box, click the play forward and backward to view the filled our rubrics for each student. Toggle the "Preview Results" button to view the field codes.
  3. Edit the layout of the rubric or modify it based on your needs.
  4. Click "Finish and Merge" -> Send Email Message on the right side of the mailings ribbon commands.
    1. The "To" field should be "M_1," which is the student's email address.
    2. Edit the subject line.
    3. The "Mail format" works best as HTML
    4. Click "OK" to email each student a customized fill out rubric.

Student Information

The 'Import Student Information' box of the "Mail Merge Rubric and Student Information' tab allows you to associate an extracted username with a student's actual first & last name and email address. To see all the extract usernames that Graderworks has found for all assignments, click "Export Student Info" and save the .csv file on your computer. The .csv file will have 4 columns:

  1. Extracted Username (found by Graderworks using the settings you provided for all assignment)
  2. Email address
  3. First name
  4. Last name

To associate an extracted username with the additional student information, fill in the columns 2-4. Also, if you anticipate more extracted usernames, you can add new rows to the .csv file. When you are finished updating the .csv file, you should save the file as a .csv file and close Excel or whatever spreadsheet program that you are using. You must close the sheet in Excel, or else Graderworks will not be permitted to read the file while it is open by Excel.

In Graderworks, click "Upload student information" and browse to the saved .csv file. Graderworks will analyze the file and output to the main window "Finished Updating Student Information" when it finishes. In the future, when you view the output Excel files, Graderworks will automatically show you the extracted Username, email, first name, and last name for all the students it knows about.


Advanced Options

Only Show Features after the origin

By default, Graderworks only shows files after the origin. If you uncheck this box, the output file will also export features before the origin. Typically, these features are based on the template the student used to make the file and are not important for grading.

Keep all of a student's files open

When grading assignments with multiple files, by default, Graderworks will keep all of the files for a single student open in SOLIDWORKS (which means they are in memory) until it finishes collecting data about all of the student's files. This method often helps speed up Graderworks since when it must analyze an assembly file, it will need to open all the part files anyway. The performance improvement is because Graderworks will not need to close the files, then reopen them to open the assembly or drawing. When Graderworks finishes grading a single student's files, it will close all the files before moving on to the next student.

SOLIDWORKS might crash if all of the student's files are open at once (for example, if a student turns in 50 SOLIDWORKS files which are all open in memory). The Windows operating system will not allow a single program to take more than an allotted amount of memory and might force it to stop. To fix this situation, uncheck the box "Keep all of a user's files open."

Output Files

Also, to exporting an Excel file with multiple sheets, you can tell Graderworks to export each sheet as a separate .csv file. These .csv files might be helpful if you are trying to read the data using some other software program that uses .csv files.

Configurations

Experimental Feature. Not well tested Some files may have more than one configuration. You can tell Graderworks how you want to grade each configuration.

Analysis

Start

Once Graderworks is fully configured, in the main window, click in the Analysis Box click "Start."

Graderworks will startup SOLIDWORKS in the background and begin to analyze the files or folders. At the bottom of the window, a progress bar will show how much work remains for Graderworks to do. Also, an estimated time remaining and expected completion is displayed. You should not use SOLIDWORKS while Graderworks is running. Depending on the number of files you are analyzing, it might take Graderworks a long time, and the information at the bottom of the window can help you estimate when your computer will be free to use SOLIDWORKS again.

Stop

The stop button stops any running Graderworks background task at the next safe stopping point.

Reanalyze and Export Existing Saved Data

If you want to view information that you previously analyzed with Graderworks, then click Analysis->Export Existing Saved Date to tell Graderworks to regenerate the output file. This action will also cause Graderworks to recalculate any file grades. You can use the feature to modify a rubric, and then to quickly recalculate grades.

View Results

Once Graderworks is finished analyzing or reanalyzing data, it will export all of its data to an Excel Workbook file with multiple sheets. Graderworks will display the output file path when it is finishing analyzing the files. Also, the output file is always in the root folder. The file naming convention is data_GRADERWORKS_year_month_day_hour_mninute.xlsx. The output file has 2 for basic usage and five sheets if you are using at least one rubric item.

  1. The first sheet in the output Excel file contains all the files and feature data about all the SOLIDWORKS files in the current assignment.
  2. If using one or more rubric item(s), the second sheet will show every file graded by Graderworks. Each row shows a different file.
  3. If using one or more rubric item(s), the third sheet will show the total assignment score (based on the weight of each rubric item). Each row shows a single student.
  4. If using one or more rubric item(s), the fourth sheet will show how each student scored on all rubric items in the assignment and show each grading item. Each row shows a single student. This sheet is designed to work well with a mail-merge in Microsoft Word so that you can email your student's assignment grades.
  5. The fifth sheet will display plagiarism information. The plagiarism check will identify all files that met the plagiarism checks and show them in this sheet. If the sheet is empty, then no files met the plagiarism detection criteria. See the plagiarism checking section for more information.

Research Usage

Are you doing research that involves analyzing SOLIDWORKS files for educational or some other purpose? We can help. We are experts in data science and SOLIDWORKS file analysis. We would love to collaborate. We can even customize Graderworks for your situation or add new features.

You can access the Graderworks data directly using the following python 3 script to get started. We recommend using Pandas to process the data.

import sqlite3
import os
import pandas as pd
pathappdata = os.getenv('APPDATA')
dbpath = os.path.join(pathappdata,'SWGrader','GraderworksDB.sqlite')
print (dbpath)
conn = sqlite3.connect(dbpath)
assignments = pd.read_sql_query("select *  from Assignment",conn)
print(assignments)
conn.close()

FAQ

  1. How can I spend more time doing XXX instead of grading my Students SOLIDWORKS files?

Read this user manual, and then use Graderworks.

  1. I found a bug!!! now what??

Contact us with as much information as possible and we will fix it as soon as we can.

  1. How do I obtain my free semester trial?

Contact us, and we will send you a license key. You can also run Graderworks with no license for 30 days.

  1. How much does Graderworks cost?

See the pricing page. Pricing is based on the number of students in the SOLIDWORKS class per semester.