Graderworks Usermanual

*

Overview

Graderworks is a grading support software for SOLIDWORKS educators. Graderworks can analyze students files, extract information, and perform routine grading tasks which will free you up to perform higher-level critiques of your students' files.


Installation

Requirements

You must have SOLIDWORKS installed on your computer for Graderworks to run.

We recommend that your computer have a solid state hard drive. Since Graderworks opens hundreds of SOLIDWORKS files, it needs to be able to read from the disk quickly which is why an SSD is helpful. Imagine how long it takes your computer fully to open a SOLIDWORKS file, then multiply this by the number of files you need to grade, and you will get approximately how long it will take for Graderworks to analyze your files. In addition, a fast processor and at least 8GB of memory is helpful.

How to Install

  1. Download the installer from the [downloads page][https://garlandindustriesllc.com/index.php/pages/view/download].
  2. For each version of SOLIDWORKS, there is a different version of Graderworks. Be sure you select the correct version when you download the software.
  3. Run the installer to install Graderworks fully. (If you already have Graderworks installed, then you will be prompted to uninstall your previous version).

Quick Start

  1. Put several SOLIDWORKS part files in a single folder on your computer.
  2. Run Graderworks.
  3. Set the root directory as the folder with the SOLIDWORKS parts.
  4. Click Analysis->Start in the top menu on the main window. Graderworks will run and analyze the files.
  5. When Graderworks finishes analyzing the files, view the results by opening the output file. The output file is in the root folder.

Basic Usage

When you first run Graderworks, the main page will show you some basic information about the software. In the message area, you will see information about your License, Graderworks version, and the version of SOLIDWORKS that is compatible with Graderworks.

Create an Assignment

You will also see information about the current assignment (in the Assignment selection group). You can add or delete assignments in addition to selecting old assignments that you have already made. To get started, you should click "Add a New Assignment" and create a new assignment.

Select the Root Folder

In the "Root Folder" group, you can see the current root folder and also browse for a different root folder. The root folder is the folder containing the SOLIDWORKS files, folders with SOLIDWORKS files, or .zip files containing SOLIDWORKS files which you wish to analyze. The default root folder is the desktop. If your student's files are not on the desktop, then you must change the root folder.

Configure the Assignment

At the top of the main window click "configure" which will cause the configuration window to open. There are several tasks that you need to perform to tell Graderworks how to analyze your student's files.

Type of Assignment

Graderworks can grade various kinds of assignments.

  1. SOLIDWORKS files in the root folder. A student can submit more than one file.
  2. Folders in the root folder. The folder name will be analyzed to extract the username. All the SOLIDWORKS files in the folder should belong to a single student.
  3. .zip files in the root folder. The .zip file name will be analyzed to extract the username. The .zip file will be unzipped into a folder with the name of the extracted username. All the SOLIDWORKS files in the .zip file belong to that student.
  4. Individual SOLIDWORKS files specified on the individual files tab in the Configuration Window. These files are not necessarily in the root folder. The Graderworks output files will still be to the root folder.

Types of SOLIDWORKS Files

Graderworks lets you select what types of files you would like it to analyze. Unchecking any of the boxes to exclude a particular type of SOLIDWORKS file.

Username Identification

In the "Username Identification" group, you must specify to Graderworks how to identify each student. Graderworks expects each student submission to have a particular format. Most schools use some learning management system (LMS) like Canvas, Blackboard, Moodle, Schoology, D2L, or some other system. These systems allow you to batch download your student submissions to your computer. During this download, the LMS will rename your students' submissions with some sort of identifier in the file name. The 'Username Identification' within Graderworks is designed to help extract the username identifier from the filename. Within Graderworks, you must specify the beginning character before the LMS added identifier, and you must specify the ending character after the LMS added identifier.

If you do not use an LMS with batch download and rename capabilities, then you could tell your students to use a particular naming convention in their file names (like appending their username and then an (-) dash).

In the table below, within Graderworks the Start Identifier is set to nothing so Graderworks will start at the beginning of the file, and the End Identifier is set to an underscore (_). The last column in the table shows the name that Graderworks will associate with each file.

Student's filename Student's school username File renamed by LMS Graderworks extracted username
box_with_hole.sldprt bob5 bob5_box_with_hole.sldprt bob5
box_with_hole.sldprt joe789 job789_box_withhole.sldprt joe 789

Once, you have selected your root folder, type of assignment, type of SOLIDWORKS files, and set the username identification, then you can preview how Graderworks will extract the usernames by clicking "preview username extraction". Graderworks will launch a new window which shows you what files or folders it found in the root folder and how it plans to extract the usernames.

The advanced section of the user manual will discuss how to map the extracted username to a student's email address, first name, and last name.

Run Graderworks

Once Graderworks is fully set up, then click "ok" on the configuration window. In the main window click the Analysis->Start from the drop-down menu at the top.

Graderworks will startup SOLIDWORKS in the background and begin to analyze the files or folders. At the bottom of the window, a progress bar will show how much work remains for Graderworks to do. Also, an estimated time remaining and expected completion are displayed. You should not use SOLIDWORKS while Graderworks is running. Depending on the number of files you are analyzing, it might take Graderworks a long time, and the information at the bottom of the window can help you estimate when your computer will be free to use SOLIDWORKS again.

View Results

Once Graderworks is finished, it will export all of its data to an Excel Workbook file with multiple sheets. Graderworks will display the output file path when it is finishing analyzing the files. Also, the output file is always in the root folder. The file naming convention is data_GRADERWORKS_year_month_day_hour_mninute.xlsx. The output file has 2 for basic usage and 5 sheets if you are using at least one rubric item.

  1. The first sheet in the output Excel file contains all the file and feature data about all the SOLIDWORKS files in the current assignment.
  2. If using one or more rubric item(s), the second sheet will show every file graded by Graderworks. Each row shows a different file.
  3. If using one or more rubric item(s), the third sheet will show the total assignment score (based on the weight of each rubric item). Each row shows a single student.
  4. If using one or more rubric item(s), the fourth sheet will show how each student scored on all rubric items in the assignment and show each grading item. Each row shows a single student. This sheet is designed to work well with a mail-merge in Microsoft Word so that you can email your student's their assignment grades.
  5. The fifth sheet will display plagiarism information. The plagiarism check will identify all files that met the plagiarism checks and show them in this sheet. If the sheet is empty, then files met the plagiarism detection criteria. See the plagiarism checking section for more information.

If you want to view information that you previously analyzed with Graderworks, then click Analysis->Export Existing Saved Date to tell Graderworks to regenerate the output file. This action will also cause Graderworks to recalculate any file grades.

Summary of Basic Usage

  1. Make a new assignment
  2. Set the root folder (the location of the files, folders, or .zip files you which to analyze)
  3. Setup basic configurations by clicking "Configure."
    1. Set the beginning and ending identifiers for your students' files (how will Graderworks know which file belongs to student which student.)
    2. Select the type of assignment (files, folders, .zip files, or individual files)
    3. Select what types of SOLIDWORKS files you would like to analyze (parts, assemblies, and/or drawings).
  4. Run Graderworks (Analysis ->Start)
  5. View Results by opening the output Excel file.

Rubric

The Graderworks rubric allows you to compare student's models to the correct answer models. Each rubric is specific to an assignment, and each rubric has one or more rubric items. Within each rubric item there are one or more grading items.

Graderworks can grade SOLIDWORKS part and rigid assembly files.

Creating a Rubric

After you have created an assignment in the main window, click on Configure to open the Configuration window. Within the configuration window, click on the "Rubric" tab. The rubric tab allows you to make new rubric items, configure a late penalty, and select to auto-launch a mail-merge of filled out rubric information when Graderworks is complete. Rubrics are specific to an assignment, so if you change the assignment, then your rubric will save and any rubric information for the new assignment will be loaded.

Creating Rubric Items

In the rubric tab, click on "Add new rubric item". Each rubric items has many options for how to apply the rubric item.

  1. Basics
    1. Name. You should give each rubric item a meaningful name.
    2. Percent of total assignment score. This field tells Graderworks how much each rubric item counts toward the overall assignment score. If you have just one rubric item for your assignment, then this field should be 100. If you have more than one rubric item, then weight each one so that the total weights add up to 100. For example, rubric item 1 might count for 35% of the assignment grade, and rubric item 2 might count for 65% of the assignment grade.
    3. File path to the correct answer. You must tell Graderworks what is the correct answer model. Browse for the model on your computer to select the filepath to the correct answer model.
  2. How to Grade.
    1. Graderworks can grade the following items.
      1. Volume. Graderworks will compare the student's volume to the solution's volume.
      2. Material. Graderworks will compare the student's material (by checking the density) to the solution's material.
      3. Composite Shape Score. Graderworks will compare the shape of the student's file to the solution's shape. This check is accomplished by aggregating several different metrics about each file.
      4. Center of mass. Graderworks will compare the center of mass of the student's file to the solution's center of mass. This grading item is helpful for grading rigid assemblies (like on the CSWA assembly questions).
      5. Fully defined sketches. Graderworks will check all sketches in the student's file and identify any that are not fully defined. Graderworks will list the underdefined sketches in the output file.
    2. For each grading item, you can select the weighted that the grading item will contribute toward the rubric item. The percentages should sum to 100.

Once you are finished making your rubric item, then click "save." You can add more than one rubric item. Graderworks will automatically determine which of the student's models match which rubric item. So if students should submit 5 files for an assignment, and if you made 5 rubric itmes, Graderworks will make sure it applies the correct rubric item to each of the student's files.

Late Penalty

Graderworks can check each file that was graded to see if the last modified date-time is after a predefined date-time which is the assignment due date. If a single file is modified after the due date, then Graderworks can take off a percentage of the student's overall grade. You must select the calendar day that the assignment is due, and you must type in the time manually to correctly set the due date and time. To use the late work check, you must also specify the percentage of points to take off.

Mail Merge Filled out Rubrics

Graderworks can automatically set up a mailmerge of filled out rubrics for you to send to your students via email or individual print and return to your students. To enable this feature, check the box in the Display Results group. The mail merge will run after you click an analysis is finished and grades are calculated (i.e. you need to go to the main window and click Analysis-> Run or Analysis -> Export Existing Saved Data).


Plagiarism Checking

Graderworks collects meta-data about the student's files including the creation date, last saved date, and saved by name. These properties are internal to the SOLIDWORKS file and are NOT the windows os equivalent. It is not possible for students to modify these properties in the SOLIDWORKS file (if you find a way, then let us know).

Graderworks uses this meta-data to check for student plagiarism. Graderworks only identifies files that meet certain criteria that are suspicious or unusual, however, what constitutes plagiarism is dependent on how you setup the assignment and what exactly you told your students. Graderworks is only a tool to help you. You are the ultimate judge of plagiarism. Not Graderworks.

Advice on interpreting plagiarism checks

A few helpful pieces of advice can steer you in the correct direction while checking for plagiarism.

  1. If you suspect plagiarism, then always check the feature meta-data. If two students turn in files turn in separate files but all of the features are created at exactly the same time (the date, hour, minute, second) by the same username, then you can probably conclude that the two files are actually the same file and that one student obtained the file from the other student. Graderworks exports the feature data in the "raw data" sheet of the output Excel workbook.
  2. Check all of the features. Often students download a part from [Grabcad][https://grabcad.com/] or some other source, then modify it and turn it in. The first few features will likely have a much different creation date than last features.
  3. Don't assume that if two students have the same "saved by" name that they shared files. They could have the same username on their computer, or they could have used the same computer. Again, the key is if both files have all or some of the features created at the same time. It is very unlikely that many of the features would be created at the same time even for students who use the same 'saved by" name.
  4. Showing the feature information (including the created by, date created, and date modified) of two files from two different students side by side is an excellent way to show a non-engineer why you suspect a student plagiarised a file. Some employees at GI LLC, have extensive experience in catching and prosecuting plagiarism in a SOLIDWORKS class, and we recommend this side by side comparison. Often in academic institutions, plagiarism accusations must be presented before some review board that is made up of non-engineers who do not understand what SOLIDWORKS is or how it is used. The review board will often understand your case if you show the feature information side by side (and the creation date and created by is the same for many of the features), tell the review board that SOLIDWORKS has "track changes" perpetually on (similar to Microsoft Word's track changes), and explain that two students could not have created by multiple features at exactly the same date, hour, minute, and second.
  5. Ignore the features before the origin when evaluating for plagiarism. Features before the origin are often based on the SOLIDWORKS template file that the part was based on. Under normal circumstances, everything after the origin is work done by the student (or a student). These features include:
    1. Comments
    2. Favorites
    3. History
    4. Sensors
    5. Design Binder
    6. Annotations
    7. Surface Bodies
    8. Solid Bodies
    9. Lights, Cameras, and Scene
    10. Equations
    11. Front plane
    12. Right plane
    13. Top plane
    14. Origin
  6. If a student is retaking a SOLIDWORKS class, then his or her file creation dates might be much older than everyone else's creation dates in your class.
  7. Checking engineering drawings for plagiarism is hard if you only check the .slddrw file. The .slddrw (drawing file) does not contain much information that is helpful for checking for plagiarism.
  8. Graderworks cannot check for some situations. For example, if a student pays someone else to do their homework for them. Although if you suspect this situation, then a pop quiz during class will likely reveal who actually knows how to model in SOLIDWORKS.
  9. SOLIDWORKS parts from the Toolbox will be very old. If you require your students to use the toolbox while making their assemblies, then you will notice that the toolbox part files were created by someone a very long time ago. If you told your students to use the toolbox, then you can ignore these files.
  10. Sometimes a student may steal a SOLIDWORKS file from another student without their knowledge. The only way you could find this out is by talking with the students.

Graderworks can perform up to five different types of checks which can help you identify plagiarism. Check the boxes next to the types of checks you would like Graderworks to perform.

More than one 'saved by' name for a single student.

If a single student submits several files then you would expect that the last savedby name would be the same for all the files. If they are not the same, then you might find this situation suspicious. Graderworks can identify these situations for you.

What if a student borrows his friend's computer to complete part of the assignment. The last saved by name could change for a single student even though the student did all of his own work. In this case, you should check the feature information. Even if they used the same computer, then the feature creation times will be much different if they both did their own work on the same computer.

The 'saved by' name is the same for different students.

If two students turn in files and the last saved by name is "bob" for both students, then you might be suspicious. Again, you should check the feature information to obtain a better understanding of how the files were created.

The 'creation date' is the same, but different students made the files.

If two students turn in files that were made on the same day, hour, minute, and second then you might suspect that one file is just a copy of the other file. You should check the feature information.

We have found this check to be very helpful and can identify most plagiarism instances when a student shares a file with another student.

The 'creation date' is less than 2 standard deviations from the average.

If a file is created a long time ago, then you might suspect that the student actually obtained the file from another source (like Grabcad). This check tries to guess a reasonable limit on the create date (average minus 2 stdev). The average and stdev is based on all files turned in by all students for the selected assignment.

If your students are modeling an assembly that requires parts from a source that you give them or from the toolbox, then this plagiarism check might not be the best. For example, some toolbox parts are created in 1998. If the student turns in a part file from 1998, then the average creation date of all the files in an assignment will be much earlier and the stdev will be very large. In these instances, a student could have obtained a file from someone last semester, and still be within 2 stdev of the average. Because of this potential issue, you might find the next check more helpful.

The 'creation date' is less than the date specified below.

Similar to the previous check, this plagiarism check seeks to identify files made outside the expected time frame. Since most files obtained from an unauthorized source were not created during the current semester, this is an excellent check to find these files from unauthorized sources. For this check to work, you must specify an oldest allowed creation date. Any file created before this specific date will be identified.


Advanced Usage

Graderworks can perform a few other tasks which you might find helpful. These options are configurable in the 'advanced' tab of the configuration window.

Export files to another format

Graderworks can create a screenshot of the isometric view of each file and save it in the folder where the SOLIDWORKS file is located or in the root folder (if you check 'save files in root folder.) These screenshots can be helpful for grading since you can open an image file much faster than opening a SOLIDWORKS file. Similarly, you can tell Graderworks to export a .stl file of each part or assembly file.

Feature

Graderwoks will analyze and export (in the "raw data" sheet of the output Excel Workbook) the feature information by default (starting in version 3.13). If you do not want Graderworks to collect or export feature information, then you can uncheck this box. If one of the rubric items is to check for fully defined sketches, then Export Feature Data will be automatically enabled.

Keep all of a student's files open

When grading assignments with multiple files, by default Graderworks will keep all of the files for a single student open (in SOLIDWORKS which means they are in memory) until it finishes collecting data about all of the student's files. This often helps speed up Graderworks since when it must analyze an assembly file it will need to open all the part files anyway. The performance improvement is because Graderworks will not need to close the files, then reopen them to open the assembly or drawing. When Graderworks finishes grading a single student's files, it will close all the files before moving on to the next student.

SOLIDWORKS might crash if all of the student's files are open at once (for example if a student turns in 50 SOLIDWORKS files which are all open in memory). The Windows operating system will not allow a single program to take more than an allotted amount of memory and might force it to stop. To fix this situation, just uncheck the box "Keep all of a user's files open."

Output Files

In addition, to exporting an Excel file with multiple sheets, you can tell Graderworks to export each sheet as a separate .csv file. These .csv files might be helpful if you are trying to read the data using some other software program which uses .csv files.

Student Information

The student information tab of the configuration window allows you to associate an extracted username with a student's actual first & last name and email address. To see all the extract usernames that Graderworks has found for all assignments click "Export Student Info" and save the .csv file on your computer. The .csv file will have 4 columns:

  1. Extracted Username (found by Graderworks using the settings you provided for each assignment)
  2. Email address
  3. First name
  4. Last name

To associate an extracted username with the additional student information, fill in the columns 2-4. In addition, if you anticipate more extracted usernames, you can add new rows to the .csv file. When you are finished updating the .csv file, you should save the file as a .csv file and close Excel or whatever spreadsheet program that you are using. You must close the sheet in Excel or else Graderworks will not be permitted to read the file while it is open by Excel.

In Graderworks click "Upload student information" and browse to the saved .csv file. Graderworks will analyze the file and output to the main window "Finished Updating Student Information" when it finishes. In the future when you view the output Excel files, Graderworks will automatically show you the extracted username, email, first name, and last name for all the student's it knows about.


Research Usage

Are you doing research that involves analyzing SOLIDWORKS files for educational or some other purpose? We can help. We are experts in SOLIDWORKS file analysis and processing large amounts of data. We would love to collaborate. We can even customize Graderworks for your situation or add new features.

You can access the Graderworks data directly using the following python 3 script to get started. We recommend using Pandas to process the data and bokeh to plot the data.

import sqlite3
import os
import pandas as pd
pathappdata = os.getenv('APPDATA')
dbpath = os.path.join(pathappdata,'SWGrader','GraderworksDB.sqlite')
print (dbpath)
conn = sqlite3.connect(dbpath)
assignments = pd.read_sql_query("select *  from Assignment",conn)
print(assignments)
conn.close()

FAQ

  1. How can I spend more time doing XXX instead of grading my Students SOLIDWORKS files?

Read this user manual, and then use Graderworks.

  1. I found a bug!!! now what??

Contact us with as much information as possible and we will fix it as soon as we can.

  1. How do I obtain my free semester trial?

Contact us and we will send you a license key. You can also run Graderworks with no license for 30 days.

  1. How much does Graderworks cost?

See the [pricing page][https://garlandindustriesllc.com/index.php/pages/view/price]. Pricing is based on the number of students in the SOLIDWORKS class per semester.