Thinkbox DEADLINE INTEGRATION

Advanced, optional configuration:

 

DEADLINE INTEGRATION

 

Configuration Overview

When renting an UltraRender chassis, it is important to me that I’m able to utilise the machine as if it were on my local network (as much as reasonably possible).

The setup I settled on allowed me to add the server to my Thinkbox’s Deadline Render queue, which meant I could manage the slaves alongside my local nodes. Renders happened seamlessly on local and remote nodes.

The only manual process involves moving files between sites; but I touch on my basic approach and a few other potential options.

Bullet points on config:

  • Windows operating system.
  • Remote Desktop management using Windows Remote Desktop.
  • File transfer via FTP.
  • Remote render management with locally hosted (our office) Deadline.
  • One or two GPUs in each server serving as a Deadline ‘slave’.
  • Properly launched 3d applications when testing scenes on the UltraRender machine.

The short ‘how-to’ listed below is an executive overview of the process and steps required to configure the above items. This is not a comprehensive walk-through, for two reasons:

  1. I’m not an expert on these topics.
  2. Your requirements and specific setup will likely require subtle (or wholesale) variations to my approach.

Lastly, you will require TeamViewer to perform the initial (optional) setup for OpenVPN and Remote Desktop, as those two applications replace the use of TeamViewer.

 

Remote Desktop using Windows Remote Desktop

The only trick here, is ensuring you’ve allowed remote connections on the UltraRender server. Go to ‘Control Panel > System > Remote Settings’ and allow remote connections.

File Transfer

There’s a few wayse to go about moving files back and forth between your office and UltraRender. For the time being, I have simply used manual FTP transfers. However, there ar alternatives which are worth considering:

  • Cloud hosted FTP.
    • Easy to setup, but very manual to manage as you need to copy each file twice (once to the cloud, then once to your office).
  • Locally hosted FTP.
    • Can more easily copy between the two locations as you only need to perform the copy once.
    • However this opens up a security risk on your server which you need to manage.
  • Google Drive / Dropbox / other similar services.
    • You could auto-sync files via gdrive / Dropbox.
    • Sharing a folder on your server for google sync might not be ideal (eg, sh aring the root folder of a large project will involve syncing potentially large amounts of data to the cloud).
    • Creating a custom ‘gdrive’ sync folder on your server is less preferable to using plain old FTP (IMO).
    • Gdrive can ‘trip up’ quite of ten on projects with thousands of files.
  • Direct drive mapping through windows.
    • It is entirely possible that you can map your local server as a Windows network drive.
    • I have found this to be VERY slow (100kBs).
    • It is possible to speed this up, but requires specialised end-to-end hardware from what little I know of the topic.
    • You could map your FTP share through Windows. I have not tried this, though it promises better speed than regular Windows drive mapping.

Deadline Farm Management

We use Deadline farm management software. If you have not heard of it, I consider it the best off-the-shelf farm distribution software available. Check it out. Configuring Deadline is as simple as installing the client, and then pointing the repository and license server to your local data.

Deadline 8’s Proxy Server allows users to securely connect to and interact with remote and cloud-based render farms over public Internet without the need for a virtual private network (VPN). (more info here on Thinkbox’s Deadline documentation website )

You will likely need to use the IP address of your data/license servers (as I’m assuming you do not have DNS working over your VPN tunnel). Eg:

  • If your license server is usually found at:
    • 27008@lixserver
    • You will need to connect via 27008@<lix server IP address>
  • The same goes for your data server. It will need to connect to the repository via your servers IP address.

The other important Deadline notes:

  • Create 1 Deadline Slave Instance PER GPU (or 2) in your UltraRender server.
  • Set the Affinity of each Slave Instance to 1 (or 2) GPU only.
  • This ensures that each GPU (or 2) is given one render task to manage.
  • I chose this option is my renderer of choice (RedShift), does not scale well across multiple GPU’s. So I treat each GPU as a separate node to avoid render speed decrease.
  • Add your UltraRender server to it’s own Deadline region.
  • Configure path mappings within Deadline, and only apply them to the UltraRender region.
  • My tip would be to mirror your folder structure within the D: drive eg:
    • If your local path is this:
      • Z:/projects/latestawesomeproject/shots/s01/maya/scenes
    • Your UltraRender path should be this:
      • D:/projects/latestawesomeproject/shots/s01/maya/scenes
    • Deadline simply needs to swap the Drive letter out and everything else is easy!
  • Once done, each node will appear on your LOCAL Deadline monitor as it’s own node! How cool is that! (very cool!).

Launching 3d apps over Remote Desktop

Remote Desktop is great. I love it. It’s the fastest, most robust remote control software I’ve used. It has one oddity however. The OpenGL graphics are not rendered on the ‘host’ machine in hardware. This means that graphics heavy applications will often NOT be able to launch via a Remote Desktop session! Houdini is one example, while Maya will launch, but not function very well. DJViewer will simply crash. To get around this, you can ‘push’ your remote desktop session to the console on the local machine, which allows opengl heavy apps to load. Once again, I’m far from an expert on these things, but I have found functional solutions to practical problems when working over VPN. And this is one of them. To launch Maya over Remote Desktop:

  • Create a batch file. Store it on your desktop. Add the following contents:

@echo off
for /f “skip=1 tokens=3 usebackq” %%s in (
 `query user %username%`
) do (
 %windir%\System32\tscon.exe %%s /dest:console
)

  • Then, to launch Maya:
    • Double click your Maya icon.
    • Very quickly, Right+Click your .bat file, run as Admin, accept the options.
    • Your Remote Desktop session will close.
    • Wait 10 seconds.
    • Remote connect to the UltraRender server, and Maya should be there in all it’s glory!

OPTIONAL – Custom Managed VPN Tunnel

For better security, I choose to route traffic through an OpenVPN server which is hosted at our office. We use a pfSense appliance for this task, and chose OpenVPN as it is easy to configure, cost effective, and also secure enough for our needs. You may prefer to use the TeamViewer VPN tunnel, which I have not had experience with. The following points relate to configuring a tunnel with OpenVPN. Ensure you have already tested your OVPN tunnel in your local environment before commencing configuration on your UltraRender server. If things are working ‘locally’, then proceed to configuring the UltraRender server.

  1. Install OpenVPN on the UltraRender server.
  2. Copy your OpenVPN config files to the server.
  3. I typically launch my OpenVPN session through the CMD prompt;
    1. c:\openvpn <path to your .ovpn config file>
    2. It is absolutely CRITICAL that you launch the command prompt as Administrator. Failure to do so will not permit the tunnel to establish itself on the server.
    3. You can do this by Right+Clicking on the ‘cmd.exe’ file and selecting ‘Run as Administrator’.
    4. Once the connection is established, try pinging your local fi leserver to ensure everything is working as expected.

Conclusion

I hope these little snippets of information at least help steer in the right direction when configuring a remote server. The next time I engage a server, I will focus on improving the file transfer process, as manually shuffling files around was far from ideal. Good luck!

Clinton Downs – NewHollandCreative

Mariusz Wolniewicz – Ultra Render