Week 15
July 2, 2017
Hi team,
This week has been about deploying the past two weeks of work into production at Mindbender, and simplifying the installation process of the pipeline for newcomers.
We’re also entering the last month or so until I expect this pipeline to be self-sustainable, which means we’ll be taking a look at what’s missing - such as Nuke and ftrack integration - and what needs work - such as the Rendering workflow.
Table of contents
Deployment
Up until now, each iteration has been small and without much trouble. But due to the size of this update, one thing became glaringly obvious.
My (remote) development environment does not accurately enough reflect the target environment.
When this happens you run into the classic “works on my machine” scenario where you are unable to reproduce some of the problems experienced on the other end. So one of the priorities for next week is (1) establishing an accurate environment, which will involve a VM of identical software and Google File Stream drive of identical server content and (2) TeamViewer access for when for whatever reason my local environment still fails to represent the target environment.
Crunch
By July 30th this year I expect each and everyone of you to feel that the functionality offered by the pipeline is enough to get you through the day.
That means taking a good look at (1) where we are at, (2) where we would like to go and (3) get there in 4 weeks time.
By then, I expect both Mindbender and Colorbleed to be dependent on the pipeline to function correctly and Colorbleed to carry their own weight in implementing features and fixing bugs. I expect to have introduced MetaPipe into the project and to have delegated part of the development milestones to them, primarily filesystem permissions and database management.
In order to get to that point I’ll need a clear line of sight between wanting to join and actually running Avalon.
Running Avalon
In parallel to the work put in by Colorbleed this week, I’ve been working on the Getting Started guide for Avalon, which meant revisiting critical functionality and simplifying its use - primarily how environment are established in order to automate the entire production pipeline.
$ cd example/project
$ avalon --build
Modeling..
Rigging..
Animating..
Rendering..
Compositing..
Editing..
Delivering..
$
The above command runs the entire pipeline, department by department, producing a final result in the exact same way artists would at Mindbender.
Having this functionality means (1) changes to any part of the pipeline can be tested against the entire stack of functionality and (2) the same content can be tested against multiple variations to the pipeline, such as where and how files are published to disk.
Most relevantly, it enables me to provide an example project alongside the installation of Avalon, so that anyone can install it and get their hands on publishing things without having to read through the manual on how to create your own content from scratch.
Challenge
One of the changes in the past two weeks caused issues for management of loaded assets, and I decided to make an example out of the solution because I suspect the problem will make itself known repeatedly as complexity continue to increase.
The problem is an old friend, and his name is Change.
One of the challenges of any software project is backwards compatibility - keeping the engine running whilst you make changes and improvements. This week we were faced with an important choice with one of two outcomes.
- An ever-growing codebase
- Continuous migration
The problem was related to how assets are loaded into Maya, where the resulting data was formatted in such a way that one of the Avalon tools - the Container Manager - could make use of it. A change was made so as to allow for more information to be displayed however this broke backwards compatibility, which meant previously loaded assets stopped working.
The responsibility of solving this is either left to the developer - me and Roy - or the artist. You.
Developer Solution
In order for us to solve it, we would need to keep two copies of code relevant to managing the data in your application, let’s call these version 1 and version 2. Assets that you load from now on would take the form of version 2, but code relevant to version 1 would need to remain in order to “keep the engine running” for current projects.
The end result is lines of code that we can never again touch, for as long as we need to support current projects in the updated pipeline.
Artist Solution
The other option is handing this “hot potato” off to you, the artists. It would mean you running a migration script whenever a change is made to the pipeline in order to enable you to keep up to date.
Problem here is that you won’t run this script on assets you no longer work on, meaning those assets will become stale and unusable if you for whatever reason need to return to it one day.
The end result is a code that only ever concerns itself with a single version of data, but data that goes out of date.
Neither approach is particularly great, but the only other option is to prohibit change.
Next Week
I’ll be tackling the final tweaks required in order to have the updated pipeline up and running on Monday. Aiming to have the Getting Started Guide finished by Tuesday and spend the rest of the week working out a plan for the following 3 weeks.
At the very least, we’ll need Nuke running and shot builds for Emre, hopefully in both Maya and Nuke.
See you next week!
Best,
Marcus