The sigh of relief on completion
Today’s article is a practical one: how to efficiently deal with the postprocessing workload and overhead, especially when you’re shooting in enormous quantity. I originally wrote this on the back of several back to back assignments and trips, but for some reason it got buried in the deluge. Now that things are a little quieter, I’ve had a chance to revisit and amend. I’m sure I’m not the only one who’s got the twin problems of shooting too much and then suffering from editing fatigue, in turn resulting in compromised selections and postprocessing…
Here’s how my March 2015 looked, and this is exceptionally busy:
– 12 teaching days
– 6 shooting days
– 6 days of travel to five countries, some halfway across the world
– 2 printing days
– Year end with closure of accounts, filing of taxes AND implementation of GST…
– Attempting to buy a house
– Still posting the full complement of 16 articles this month
In short: up to the gills. Effectively, there were about fifteen full shooting days in there; that’s actually a lot, because I find a good rule of thumb is at least one day – and sometimes two in the case of heavy retouching – for editing and postprocessing after each shooting day. This is why even saving a few seconds per image is worth chasing for me, with the net result being PS Workflow II. From those fifteen days, somewhere in the neighbourhood of 10,000 images fell out; some of which required quite a bit of postprocessing – mainly retouching or manual control point stitching of images for my upcoming Forest exhibition.
Doing the basic math, it’s clear to see that even if you take just one second to view each image, it’ll take about three hours just to see everything once. And it will definitely take longer since you also have to spend some time on evaluation and checking of technical properties before deciding what to process and what not to. Editing – the process of curation – has now become both time consuming and critical to manage one’s workflow. For the client shoots, fortunately the client does the selection, with some suggestions from me. They’re also generally easier as either there is a shot list in the case of studio work, or there’s a story objective for corporate documentary. There, the major time consumer is retouching or client communication/turnaround – especially if they’re in a different time zone. I simply keep every file shot, just in case. You never know when clients may need a different angle or want an alternative or merged image.
The much more difficult thing to manage is personal work, either shot during my own time between jobs on the road or when I’m teaching; here, I experiment a lot, and only discard the obvious failures (gross composition, focus, exposure) from viewing the camera’s LCD. The net result is literally hundreds of gigabytes of data. I know a lot of my students had a similar challenge after the Prague Masterclass: lots of shooting, lots of experimentation, an enormous deluge of images to curate. On top of that, I did landscape work – including large stitches – for three days afterwards. Live working backups are the first order of the day, until the final processing and curation is complete. I will copy cards daily to a pair of hard drives in the field; different brands for reliability, and keep the files on the original cards until I’m ready to archive. Each shoot or job or location gets a new card. I’ve got somewhere in the region of 700GB of memory cards – and in a month like this one, I’m actually going to get through most of them before I can start archiving.
Working drive space management now becomes critical. I’ve got about 200GB or thereabouts, but lately have had to start using an additional 500GB 1TB USB3 SSD – the intermediate TIFF-16 files for one stitch may be as much as 20GB alone.
I’m finding more and more that incremental curation is both efficient in weeding out images that aren’t portfolio-grade, and helps to reduce overall working time: basically, with each pass, take a few out that obviously don’t work. The more time between capture and curation, the easier it is to be objective. But just don’t leave it too long because it’s also easy to forget the intended final presentation. I’ll keep up this curation process until the point I’m done processing everything; inevitably there will be images that you thought you could process to a final result, but for whatever reason turn out to be unworkable. A 99% discard rate is not uncommon for me for personal work. The most difficult decisions to make are for files that are very similar, but involve some minor compositional changes between them; on any given day you may feel one is better than the other and regret your decision in the long run. Often I’ll process both to the best possible endpoint for each image and see which I prefer then.
This may seem obvious, but you should ideally curate and post process on the same monitor – given that some culling decisions are driven by postprocessing or latitude or file potential, it doesn’t really make sense to do it any other way. Even though consistent monitor calibration can homogenise things to some degree, most of the time we’re curating in the field on a much more restricted gamut laptop and processing on the desktop. You simply can’t make a good decision based on limited information.
Despite what I’ve said about the extremely (~45% ARGB) gamut of the 11″ MacBook Air’s monitor, I’ve done a surprising amount of processing on it simply because it’s what I’ve got with me when I’m waiting in an airport or on a plane for 12 hours. I’ll make sure I finish that set on the same machine though, even after I get home – otherwise, it’s far too easy to forget to miss a file or waste hours in transit especially if your seat happens to have power. I’m finding myself carrying the computer around with a small subset of files to work with (there are obvious space limits on the Air’s hard drive) in case I get a few spare moments; slowly, the backlog gets cleared. It’s tempting to change machines to a 13″ Retina MBP or 12″ Retina Macbook for better gamut, but the density of the display actually makes it difficult to assess critical sharpness and pixel-level acuity – not to mention missing very minor things that have to be retouched out but land up getting seen by clients on non-Retina displays. Update: this article was written a little while back and got bumped forward because of the E-M5II review. I am using the current 13″ rMBP with the display set to either 1680×1050 for better eye relief, or the full 2560×1440 for critical work – I just go very close to the screen.
The only thing to do is check final files on the main machine’s 27″ Thunderbolt Display afterwards, of course – there are a lot of colours that are simply out of gamut for the smaller screen. This is especially critical when proofing for print, for which I’ve got another color profile that represents the paper and printer I’ll be using to the best ability of the monitor; that said, there’s still no substitute for the physical thing since there’s a limit both to the frequency of information the monitor can display, and some mismatches towards the edges of the gamut – most noticeably in the very saturated reds (prints are better) and the light blues (monitor is better).
Personally, I prefer to have as unbroken a processing session as possible, especially if consistency is paramount – for client files or for a set that will be presented together. This is obviously tough if the set is very large, so my curation will both be quite brutal as well as grouping similar files together using the labels in Bridge so I know what should be processed together. Balancing continuity with making full use of the time available is the tricky part. And worse when you’re jet lagged; somehow I find that waiting for files to open tends to send me to sleep…
When there are no final raw files left, I’ll go back to the start and make sure a) there’s the same number of input and output files and b) now that I’ve got somewhat fresh eyes after not having seen the initial images for a while, make a final cut. Only then do things get archived offline to RAID arrays and cards wiped clean for the next round. Anything that has an onward destination then gets sent to clients or uploaded for online use.
Here are the takeaway tips to make your postprocessing as efficient and consistent as possible:
- Curate continuously, and critically. Do a bit at a time many times rather than all at once – it’s the best way to average out your objectivity. Continually remind yourself of the end objective.
- Process the standout singles first – this way, you’re both freshest and spend most time on the best images. It’s inevitable that after thousands, a certain degree of jadedness sets in.
- Don’t delete anything until you’re ready to archive the whole set; you never know when you might need to go back for an image that fits the story better even if it isn’t necessarily the strongest one as a standalone.
- Try to be as consistent as possible – use the same machine to curate and process since you’re not going to be able to process everything before curating it.
- Try to automate whatever you can – batch conversion of RAWs to TIFF 16s for stitching, the stitching itself, etc. The computer can do this while you sleep; it doesn’t need to. One thing I did come away from this realising is that I need even larger scratch disks – apparently 500GB isn’t enough for multiple 16-bit gigapixel-plus images.
- Curate again after processing on another device to check your postprocessing doesn’t dominate over the image or isn’t sufficient to support it.
- Use a consistent but flexible workflow so your final output from a shoot can stand together.
- Keep a master processing folder somewhere; for me, as it gets larger and larger, it’s starting to live on an SSD on a SATA-USB3 cable so I can use it on any machine and get stuff done on the go. Similarly, keep as much of your live processing as far as possible on a fast pipeline – SSDs make an enormous difference, and are now cheap enough that there’s not much excuse not to use them.
- Process similar images together: both to make it easier to assess what to keep and what to discard, as well as to ensure output looks visually consistent from a stylistic perspective.
- Backup your archive.
I have to say, it’s difficult to beat the satisfaction of an empty ‘Processing’ folder – usually, there’s just enough time (typically a day or two) to enjoy it before it starts filling up again…MT
The process of curation is covered extensively with a large example set in PS Workflow II.I recommend the Sandisk Extreme and Ultra SSDs; a 960GB Ultra II SSD with 550MB/s read-write is under $380, and for those of you with smaller files or less volume, you can get a not much slower Ultra Plus 256GB SSD for about $120.
__________________
Turn the mood and style up to 11: the Hanoi Cinematic Masterclass in association with Zeiss (21-26 July and 28 July-2 August inclusive) is now open for registration – click here to book and for more information.
__________________
Visit the Teaching Store to up your photographic game – including workshop and Photoshop Workflow videos and the customized Email School of Photography; or go mobile with the Photography Compendium for iPad. You can also get your gear from B&H and Amazon. Prices are the same as normal, however a small portion of your purchase value is referred back to me. Thanks!
Don’t forget to like us on Facebook and join the reader Flickr group!
Images and content copyright Ming Thein | mingthein.com 2012 onwards. All rights reserved
Ming Thein | Photographer 
whoa that is busy. Sounds like you need an assistant to do the non important stuff so you can fit more important stuff into your life… like relaxing, family time. I’m available if you want someone committed!
You live in the wrong country. I haven’t been able to find somebody halfway competent in Malaysia who wouldn’t add work for me to redo things afterwards – worse still, a waste of time and I’d have to pay for the privilege…
I would strongly encourge folks to look at Photo Mechanic. Absolutlely fantastic program. MUCH better than LR for ingesting, viewing, culling, metadata annotation, etc, IMHO. (I have no affiliation; just love the program)
But you’d still need to use another raw converter/ post processing software, right? I’m not sure adding another step is faster. You can configure bridge at least (not sure about LR) with different metadata profiles that can be automatically written.
Another vote for Photo Mechanic. Its speed in editing large quantities of RAW files, tagging/colour coding, moving files, adding IPTC etc. is incredible. Like you, I come back from assignments with 1000-1500 RAW’s to edit down for client selection and I have not found any other software anywhere near as fast. Once I have the final edit, I just open them all in ACR, work on them, and save as JPEG/TIFFs from there. Try a free download copy, every Pro I know in the UK who has the amount of files I seem to generate seems to use it.
I’ve also been trying to reduce the photos I keep by making multiple sweeps through the images I recently captured. This seems to run counter to what mainstream consumers are being encouraged to do with new services like Amazon photos, Google photos, etc.that offer unlimited storage and automatic grouping–over record and under curate.
I notice in your post that you are back to using the 11” MBA–did you have second thoughts about the 13” rMBP? I’m considering the 11” myself, especially since 1) I don’t print and 2) you can get a maxed-out version for around 1200 us dollars on the apple refurb store…
Thanks again for all the hard work you put into the site.
There’s just no point in making more images nobody is going to see – photographer included.
Oops. The post was written when I had the 11″, and delayed because of the E-M5II review. I’m using the current 13″ rMBP now.
Just mind-blowing!. Thanks for the insight into your work life and the tips.
No problem!
Impressive output Ming. You are there where seconds counts. Might sound uber-stressing, but knowing you, you appear uber-ralaxed and have time for the random to happen.
I have made a shortcut through it all. Very tight curation leaves me with very little post work to do, slim my hard discs, .. and the tighter I can make it the better is the result. There’s also the fact that even your best work might be forgotten a moment after viewing it, so why show too much of it all. I need to print my best stuff in order to enjoy in full.
I’d actually rather claw the seconds back because they add up…more time for printing!
And the prints is how I want to view the photograph ultimately.
Great post, I’ve been struggling with the motivation lately to process my ever expanding collection of travel photos.
Funny thing is I up until earlier this year a good part of my job was processing and retouching images (weddings, events, school photos) but when it comes to my own work it’s much harder to knuckle down and focus on it. I’ve reached a point now where I’m just doing the most basics and uploading simply to share with friends who mostly only interested in seeing where I’ve been than portfolio-grade images.
I think I really need to sit down and seriously curate my work to put together a smaller, stronger collection of my favourites though.
It’s when work turns into a hobby – or vice versa. I think curation is actually a very good solution as it minimizes the number of images to process AND results in a stronger final set.
Do you think people interested in photography (but not currently in it) understand the amount of post work required for results they want? I don’t think they do, and I think if they knew they’d lose interest quickly.
That’s one area where iPhone kills “professional” equipment. In the hands of a not-professional, iPhone will give better results because iPhone has lots of smart software in the background making sure things turn out punchy. Or there’s a thoughtless filter.
Post processing is probably the biggest area for improvement, but none of the traditional camera manufacturers are tackling it. It’s all so clumsy now, both workflow and the software we use. The current state of it is no place for casual photo shooters looking to up their smartphone game.
Well, it’s not a lot of post work per image – I wouldn’t be able to produce enough work if it was – it’s just establishing a consistent and meticulous workflow that’s tricky.
I don’t think post processing is avoidable as here’s no way a) all situations can be standardized and b) the software people can read our minds…
That photograph is excellent. Great image. Intellectual composition is an inspiration.
Thanks!
Great tips and congrats for making it through all that! I find headphones linked to upbeat music also help 😉
Absolutely can’t argue with that, though it’s speakers for me these days as I need to keep an ear out for the baby…
Congrats on that!
I’m exhausted just reading your post! 😉 I may just give up photography and start looking to be your sole supplier of coffee.
In all seriousness, a great post, especially as I sit here and look at 5 five needing to be processed.
I’d say thanks, but you only reminded me about my backlog, and worse yet, my shortage of available time. But I’ll say it anyways.
Thanks,
–Ken
I could use a sole supplier of coffee if he can get me a good quantity of Brazil Santos…
I wish I just had five. I did about 90 before breakfast and still have several hundred to go – must clear them before the end of tomorrow as I go on to the next job…
Sorry to have to spoil your taste buds ( trying to help you be healthy )
Caffeine destroys nutrients – vitamins & minerals
According to John Hopkins hospitals latest discovery ( inventor of chemo therapy )- its the lack of vitamin & minerals that makes one cancer prone
My own obsevation – people who look like my uncles ( greying ) are actually 10-15 younger than me !
( and on checking are coffee drinkers )
My hair is still natural black – i dont consume coffee/ tea / chocolate/
My bio age tested to be 40 – i am almost 70 !
Hmm, I’ve got to find another way to stay awake then…
Ditch the cigars, enjoy a cup of coffee.
Why not both? :p
Better alternative to caffeine
Upon being awake >
A) first thing – rinse your mouth then consume the juice of half a fresh lemon with at least three glasses of 8 fluid oz of water
( cold or room temp but not hot ) the remaining half lemon is for your good wife / family !
1> You will feel instantly refreshed / invigorated similar to having consumed coffee ( better ! ) without having the side effects of the caffeine destroying your calcium / vitamins & minerals that would make you feel worse later & eventual premature greying plus aging / finally cancer !
2> No – you wont get gastric as the lemon juice would turn your blood alkaline ( ideal healthy condition )
3> You would within two months of diligent daily consumption enjoy over a dozen health benefits !
4> You would also notice bowel movements are now easy – no more constipation !
5> the vitamin c also prevents scurvy !
6> your complexion would soon become much better & fairer !
7> you would no longer feel as sluggish as it would help cleanse your blood vessels from unhealthy vegetable oil ( choking palm oil ) that the majority of food vendors now use ( that cause hair loss / heart issues / sudden death )
8) no more leg cramps ( when we exercise vigoriously our body produces lactic acid that cause cramps / lemon juice destroys lactic acid )
9> intially when you no longer consume caffeine ( coffee / tea / chocolate / soft drinks – they added caffeine to get you addicted)
you would experience headaches from withdrawal pains ( caffeine is a drug )
But dont worry you would soon get over it & enjoy one great benefit – calcium is no longer being leached from your body – another cause of leg cramps from lack of calcium / broken bones / hip fractures / spinal fractures ) by the way get your calcium from large kale stems that does not cause kidney stones unlike inorganic calcium suppliments ( chalk )
10) Because you no longer drink coffee sweetened with refined sugar – you are less susceptable to becoming diabetic
( If you are already diabetic – consume cinnamon powder with your food – the cure ! )
11> if you have flaky skin around your elbow – just rub the inside of the lemon on it before you decide to discard it – no waste !
12> excess body fat would soon melt away !
13> no more middle age tiredness or need for afternoon naps !
14> saves you money (cost less than coffee or tea or chocolate / hospitalization bills )
thank God- he created lemon for our well being !
GodBless !
bonus > consume a handful of pistachio nuts ( contains 31 vitamins & minerals ! ) instead of typical man made artificiial vitamins & minerals that cause blood vessels hardening / kidney stones / irratic hair growth/ etc )
Righto. I can’t say that 4, 7, 8, 9, 10, 11, 12 or 13 are issues though!
There was a typo in my post. I meant to say 5 full cards. As far as coffee, I have no connections in Brazil. But I did meet a few growers in Kona if that type of bean is your cup of tea (pardon the mixed metaphor pun).
–Ken
Hah! Just kidding. I have a pretty decent local supplier. Oddly I drink less overall now that I make it myself…I suppose one’s expectations have changed.
Nice and clear! I like it!!