Deadline for Maya

Render Farm Preparation

Before you go about submitting your project to the render farm, consider these points:

• Keep render resolution 1920x1080 or below. Unless you need a larger frame for compositing purposes, please keep your renders to HD. 4K quadruples render times over HD.

• Remove spaces and special characters (<>:"/\|?*) from folders and file names. Change "My project" to "Myproject" or "My_project" and "my file.ma" to "myfile.ma" or "my_file.ma". (<>:"/\|?*) and other special characters can interfere with how a file name is "read" by the computer, so it is best to avoid them entirely. Limit file names to alphanumeric characters (abc..., 123...).
  

• Remove any extra geometry. Only objects that will be directly in front of the camera need to be present. Do not hide objects, delete them from the scene. This may involve creating a new scene for each of your shots.

• Clean and simplify your scene to the best of your ability. In Maya, consider running File>Optimize Scene Size with "remove duplicate shading networks" and "remove unknown nodes" checked on. Go to the Hypershade window and go to Edit>Delete Unused Nodes to remove unused materials.

• Cache simulations. If you have simulations (fluid, cloth, hair), you need to bake any simulation to a cache file. This will solve jittery hair and cloth. Go here for help on how to create an ncache: | Create new ncache. Note that if you make a change to animation that the simulation relies on, the cache will have to be recalculated. Hint: use mcx and 1 file per frame.

• Save revisions of your scene. Save a “render” version that’s been cached, imported references, and optimized, and a “live” version for continued editing.

• Ensure ALL project media, referenced files, textures, caches, and other files needed for rendering are placed within your project structure.

• Optimize your scenes regularly. If you made changes since the last optimization, then you should optimize again.

• Avoid using unnecessarily high-poly models. Large amounts of polygons can add a lot to render times.

• Use only as many lights as absolutely necessary. Lights significantly add to render times.

• Keep file names under 256 characters.

• Check all render settings. Ensure your render settings are at production quality, but not so high as to cause inflated render times. For Arnold renders, this means not setting the Camera (AA) sampling rate too high. 3 or 4 is good for production quality renders. The other sample rates (Diffuse, Specular etc...) should be adjusted as needed according to the needs of your scene. Check out the Arnold documentation for useful information regarding what each sample rate is for here.

• Do not render in jpeg's. Jpeg uses lossy compression, meaning it does not retain all the data in your image. It also degrades every time you save it. Use png's or exr's (or any other format that is lossless).

• Don’t set reflections to be too deep. Don’t make mirrors that reflect mirrors that refract through glass that reflect off mirrors, etc. Tune the reflection depth to what each scene needs.

• Consider converting any .psd textures to an efficient flattened file format like png, tif, tga etc. Use large 4k textures only when necessary, otherwise stick with smaller 2k, 1k, or 512x512 textures.

• Consider breaking your scene up into layers, isolating moving and non-moving elements.

• Use batch render to test render times. Note that initiating a Maya viewport render will not always accurately gauge your render times if you have any render layers or AOVs / Render Passes active. The Maya viewport render only renders one layer at a time and one AOV at a time. If you really want to see how long a frame is going to take you have to batch render a few frames. 

• Import references. Optional but recommended. Importing your referenced models and then optimizing your scene is easier than optimizing your references one at a time. Be sure to save a new version once you import your references! Use the file with imported references to render and continue editing in the primary file.
• Aim for 20 minutes maximum per frame: Long render jobs can tie up the whole farm for days.
• Redshift VS Arnold: Arnold is popular, but Redshift is faster, especially with our dedicated GPU render server with a rack of 8 RTX 6000 GPUs.

Submitting a Job

Step One - Login to your personal account or PCoIP

• Click on "Other User" and login to your personal account, using your myEC credentials (first initial lastname, myEC password).
• You can submit to the Render Farm over PCoIP. 

Step Two - Copy All Project Materials To Render Drive

• Maya projects should use the default project folder organization created via File/Project Window.
• Navigate to the Render Drive: Open Windows Explorer and click on the computer name with your username appended to it, or click the drop down menu. Click on Render (R:).
  • The Render Drive has changed format; Each user now gets their own directory, so you will only see your own files. This eliminates the clutter of the dumping ground that used to be the Render Drive.
• Move your entire project folder (minus any unnecessary renders that might be in the ‘images’ folder) to your folder on R:\.
  • If you are using PCoIP to submit your render, then use OneDrive to transfer your files to the remote computer from your personal device. You get 1 terabyte of cloud space with OneDrive through ECU (login is also with your myEC credentials).
• Refer to the images below for clarification.

Render Drive location in Windows Explorer.                                                                   Default Maya project

Step Three - Open Project From Render Drive to Verify References, Paths & Render Settings

• Once your files have successfully copied to R:\, use File>Set Project to set the project directory to the project folder on R:\., then open your scene.
• Verify that texture and other references are pointing to the proper, relative locations on R:\ & that all referenced files have indeed been copied to the render drive. Any lingering references to files on C:\ or S:\ will cause your render to error. Go to File>Reference Editor to point any referenced files you may have to the correct location on R:\. Another option would be to import all of your references into your scene. This will eliminate possible reference errors, though be sure to optimize your scene when you do this, and save it off as a different version.
• Missing textures is the most common error with the render farm, so please check all your file paths carefully. TIP: in the hypershade, use Edit>Delete Duplicate Shading Networks and Edit>Delete Unused Nodes to remove excess materials and cut down on clutter. This will remove any materials not actively being used in your scene. Once you have done that, use the File Path Editor (Windows>General Editors>File Path Editor) to find any missing textures or references.
  
• In your render settings, ensure that your output (e.g. ‘images’) is also set relative to your project folder on R:\.
• In your render settings, ensure that you have selected an image sequence output, rather than a video file like .mov or .avi (Video renders not supported by network rendering).
• Check that your file padding (file padding refers to the amount of zeros you put on the frame number) & extension order is appropriate: do not put the frame numbering at the end of the filename. This will break the file since the computer will try and read the numbers as the file extension. For Redshift users, although you cannot change the order, the default it gives you is functional.

Step Four - After making any changes ensure that you re-save the file to R:\.

Step Five - Submit Job from Maya using Deadline tool tab.

• With your project open, open the Deadline tool tab and hit the Submit Job button. This will launch a separate Deadline submission dialogue that contains your job details auto-filled based on project settings & best practice defaults.

Deadline submission button

• Name your job something descriptive, to keep your renders organized (cstrickler_sc01_sh03_group_scene_v02). Use v01, v02, etc... if you need to re-render a scene
• Enter your frame list and set the camera.
• Double check that your scene file & project directories are correct & relative to R:\. If you see C:\ or S:\ or any other letter at the beginning of the path, your render will not work.
• Hit the "Submit Job" button.
  

Deadline Submission Window

Step Six - Manage your job

Once the job has been submitted, you can open the Deadline application "Monitor" to keep track of your job and about other activity on the farm. To open Monitor, you can find it in the start menu under "Network Rendering", it has a green computer monitor as an icon.

Deadline Monitor

• From here you can keep track of your render. The left panel is a list of all the jobs currently rendering and completed. If you click on your job, the panel on the right will let you know information about each frame.
• Within 30 seconds your job should change from ‘queued’ to ‘rendering’, unless there is a long queue on the farm.
• Your job may gather (many) errors, but most errors are a computer that tried to pick up your job but was unable to render because it was missing a plugin or was interrupted by someone needing to use that computer.
• If your job does not begin after a few minutes or gathers many errors without making progress, double check your image paths and render output and make sure they all are on the R:\ drive.
• Notice a particular computer is consistently messing up your render? You can right-click on a frame rendered by the culprit and select "Requeue Tasks by *computer name*". A dialogue box with a checkbox will come up and ask if you want to blacklist the computer. Click the checkbox and hit "OK" and all tasks previously rendered by that computer will be redone, and that computer will be prevented from rendering your job again. If this happens, please email us at animtech@ecuad.ca so we can take a look at that computer.
• Do not delete jobs, we can use the error reports to troubleshoot what may be wrong with your render.

Step Seven - Retrieve your files

• Once your job is completed, you can retrieve your files from the images folder of your project folder on the Render Drive.
  • If you are on PCoIP, you can transfer your image sequence over OneDrive, or simply use the PCoIP computer to create a video that you can more easily transfer to OneDrive or upload to moodle, Youtube, Vimeo, etc. . Make sure to back up your work.
• Please delete your rendered images and project folder once you are absolutely sure you don't need them anymore and have them backed up in other locations. We need to keep the Render Farm clean and tidy in order to ensure it runs smoothly. All files will be deleted from the Render Drive in May.

Failed Jobs

If your render is slow or producing undesirable results, you can halt your process in one of a couple ways:
Right click:

• Suspend - This will pause the current job, but allow frames that are currently rendering to finish.

• Fail Job - This will pause the current job, stopping all rendering frames.

Main Reasons Render Jobs Fail

The 5 most common network rendering errors in order of frequency

1. Files and folders are not properly placed in the R:\ Render Drive. This is by far the most common error. Please check your textures to make sure they are in the right location. This is managed much more easily if you have a properly set up project folder.
   
2. Incorrect project paths, even if your files are in the correct place (e.g. the texture paths in your scene have C:\ instead of R:\).
3. Missing project files, especially textures.
4. Missing plugins/scripts: Emily Carr must own all the plugins you are using , or be able to download them for free! Cracked plugins will not work!
5.  Inefficiently or bizarrely constructed scene files. Too many lights or needlessly complicated scene geometry are common mistakes.
   

If the job keeps failing after you’ve ruled out all these things, please email animtech@ecuad.ca