Quality control, sample variation and what it means for photographers

_1M10077 copy
Perhaps the most problematic series of lenses I’ve ever owned. Like the fable, when they’re good – they’re really good; when they’re bad, they’re horrible.

An increasingly heard phrase amongst photographers and gear collectors is “it’s a good copy” or “it’s a bad copy”: today’s article explores what this actually means, as well as how it is relevant in real terms.

First, a little background: for the vast majority of my recent lenses, I’ve owned or tried more than one sample. In some cases, quite a number of samples. Not all of them perform equally well; the better ones do, but there have been some enormous variations between good and bad, too. The reason for this goes back to the manufacturing process: anything that is made or assembled from components has tolerances; the dimensions of an object are not 10x10x10mm, they’re really 10+/-0.1 x 10+/-0.1 x 10+/-0.1mm. The magnitude of the +/- portion – the uncertainty or the tolerance – depends on the manufacturing process itself and the quality control procedures employed. The tighter the tolerances, the more expensive and time consuming this process is: remember, measurement itself is not a simple thing; your tools need to be both sufficiently precise and accurate to begin with, otherwise they simply cannot serve as a useful reference point.

Bringing things back to the price of eggs, this means your lens element might vary from 9.9×9.9×9.9mm to 10.1×10.1×10.1mm, or anything intermediate. Clearly then, you can see how this might be a problem with precision optics – fortunately the tolerances are of course a lot smaller; +/- 0.1mm was just an example. But the same thing applies not just to the shape and dimensions of the components, but the assembly and alignment of those components, too – maybe the individual elements need to be centred to within 0.05mm to be ‘acceptable’; you might have a whole bunch that are individually out by 0.05mm, which might create very bad astigmatism – but still be ‘within tolerances’; or you might have every element dead centre; or you might have some odd combination of skewed elements that cancels out decentering, but isn’t that sharp. Needless to say, the more elements and components inside a system, the greater potential for something to go very wrong*. A combination of all of these tolerances coming into play is what makes the difference between a lens reaching the theoretical maximum resolving power and being just a little bit soft.

*It’s for this reason I’m against the use of adapted lenses: the same tolerances of thickness/ size/ planarity/ centring etc. apply to the mount surfaces, too; regardless of who makes them. Introducing two more mounting surfaces into the equation might or might not create undesirable effects, but it will certainly increase the chances of it happening – and given the resolution of today’s camera, it will be more obvious and easily detectable, too.

Any component whose position and dimensions relative to another component are susceptible to this, and affected by even the slightest ‘miss’; whether we can detect it or not is another question: until recently, resolving power of the camera side of things wasn’t really sufficient to detect anything but the most serious cases of out of tolerance or misalignment. Consequently, manufacturers were able to get away with much looser quality control procedures. I suspect that tightening QC is partially responsible for the recent increases in lens prices, though it could also be arrogant greed or increasingly complex optics in some cases.

The D800E is a good case in point: together with the A7R, it is perhaps the most demanding camera at the moment in terms of resolving power. (The A7R is actually more demanding of the assembly side, as its mount spec requires less helicoid movement; therefore an identical absolute change in helicoid position will affect the D800E less than it will the A7R – with the consequent impact on resolving power due to shifting focal plane.) Case in point: the tolerances for this camera are so tight that when I check my AF-fine tune settings every 20,000 exposures or so – i.e. 20,000 shutter cycles – I find inevitably that things have drifted slightly; my guess is that the mirror moving/ wearing in/ coming loose/ whatever is going on – affects the sub mirror position enough that you need to compensate autofocus to obtain optimum results.

Remember the left side focusing issue with that camera? I’m willing to bet that the misalignment of the AF sensor module (or AF sub mirror, hard to tell which) was so small that it would have been a non-issue with a D700. Beyond the camera itself, lens selection matters: I went through five samples of the AFS 28/1.8 G until I found one that didn’t exhibit any decentering. Slower lenses tend to be more forgiving because depth of field masks imperfections to some extent. Fast, long lenses also mask imperfections because it’s very, very rare that you’ll have all parts of the focal plane at the same distance and in focus anyway – so you simply won’t see if some portion of the frame isn’t pulling its weight.

Granted, my threshold tolerance for imperfection is probably lower than most, but there’s no point in buying a lens that doesn’t do the job. And the more expensive the lens, the more performance I expect – be it maximum aperture or resolving power or some other quality. Unfortunately, the higher the theoretical performance, the tighter the tolerances must be to achieve it. This means that realistically – automated assembly with human QC is preferable to human everything; a machine can be both more precise and more consistent – providing it is properly calibrated to begin with – than a human. But it’s much easier for a human eye to spot deviation from pattern than a machine; it’s just one of the things our brain is good at (remember subject isolation and breaking pattern?) Of course, if you have an assembly line that isn’t properly calibrated, you get a very accurate reproduction of an imprecise object – early D800E AF modules again being a case in point.

My experience with hand-assembled cameras and lenses – namely, Leica – has been less than stellar. I’ve gone through six samples of the 50/1.4 ASPH; one was mechanically defective – it threw an aperture blade on day two – one was astigmatic; two were just soft – one I suspect had a slightly too-short intermediate helicoid, the other perhaps elements that skewed slightly in all directions; only the first and last were ‘good copies’ – i.e. elements were aligned, the mechanical bits didn’t break, and the lens generally performed to expectations and in line with the theoretical MTF chart**. The thing is, short of the broken aperture – if you didn’t handle more than one sample, you wouldn’t know that the one you have was defective. And this is how I became very conscious of sample variation in the first place: I made the mistaken assumption that all of the lenses were like my first one, which I sold to a friend knowing that there was another one sitting at the dealer.

**Note that almost all MTF charts are theoretical maximums calculated by computer; as far as I know, only Zeiss actually use measured results for their published MTF charts. So what you think you’re getting might not necessarily match what you actually get.

You can of course use this information in two ways: test the actual lens you’re going to buy, and if you’re happy with it, then don’t try anybody else’s lenses – ignorance being bliss and all that. Or, practically, make friends with your dealer so you can try a reasonably large sample and pick the best one. I try a reasonably large sample and talk to people I know who evaluate lenses on the same criteria I do. Here’s another thing: in order to produce a meaningful review that applies to the majority of samples – whether it’s a lens or camera – it is necessary to try multiple units to ensure that results are consistent. I mention it now because it occurs to me that few, if any, ‘reviewers’ do this; what it means is that their conclusions may only apply to that single unit. If it applies to a large sample of the population, then it’s probably a trend; if not, then it’s meaningless – it could be bad luck. I’ll only mention a problem if I’ve found it on the majority of units tested, and the number of units tested is quite a few – again, not to harp on the D800E issue – but I tested eight units, all of which displayed the problem. Similarly, I tried two copies of the Zeiss Otus (at the time, a full 1% of the total production, and all I could get my hands on) and checked my results with other owners before coming to the conclusion that it was perhaps the best lens ever made for F mount.

I want to finish by putting all of this into perspective: in general, you’re not going to be able to see much difference on systems below 12MP. Current production tolerances are sufficiently tight that you can shoot with confidence. 24MP, and especially on higher density sensors, is more demanding; and the 36MP sensors are the worst of the lot to date – eking out every last bit of information from these is a task that is going to require climbing very far up the diminishing returns curve indeed. In my experience – both photographing and evaluating student’s files – the transition from 16 to 24MP is where you start to consistently start to see the effects of sample variation, but beyond that, it’s also the transition point where technique and shot discipline become dominant limiting factors. So, in conclusion: sample variation matters; pick the best sample you can; but before you spend too much time and effort worrying about it, make sure that your technique is good enough to consistently see the difference in the first place…MT


2014 Making Outstanding Images Workshops: Melbourne, Sydney and London – click here for more information and to book!


Visit the Teaching Store to up your photographic game – including workshop and Photoshop Workflow videos and the customized Email School of Photography; or go mobile with the Photography Compendium for iPad. You can also get your gear from B&H and Amazon. Prices are the same as normal, however a small portion of your purchase value is referred back to me. Thanks!

Don’t forget to like us on Facebook and join the reader Flickr group!


Images and content copyright Ming Thein | mingthein.com 2012 onwards. All rights reserved


  1. I think that we all forget, that cameras and their very expensive lenses, are not professional calibrated measuring tools! If I would own for example, an Zeiss OTUS lens, I would like to spend that extra money, to make sure to get both, the camera and lens, calibrated from Zeiss! Another weak factor, are the, usually with the camera body delivered focusing screens. They are good enough for quick autofocus, but for sure, not for an precise manual focusing! Another big weak factor are the eye capacity of the individual photographer, wearing glasses nor not! I guess, there are not nearly 100% to obtain, with the most expensive optical glasses either!

  2. Seems like an opportunity for someone to offer qualification as a service. If your willing to spend $1000+ on a lens, a small fee to a company to determine copy quality (especially within the return period) would be a prudent investment.

    • I’d be inclined to agree, but by the time the logistics and insurance are factored in, I’m not so sure.

      • learnerlang says:

        Perhaps a large fee then, what price for peace of mind?

        • Only the buyer can answer that. You’re supposedly already paying a large premium for Leica QC, but in my experience ‘handmade’ translates to ‘very large sample variation’…

  3. Thank you for a wonderful confirmation of an issue some of us have raised and numerous folks have discounted.
    My biggest complaint is the impact finding the “good copy” has on relationships with online retailers. There are no stores left in the places we live. We literally have had to try more than 20 copies of a various Sony lenses to find one of each that was not decentered. All were multiple purchases from several dealers after telling them we would be keeping one, if it was a “good copy” In all cases they willingly sold us the lenses. They also refunded my $$ when returned. However, two major retailers now refuse to sell me anything unless it is on a “no return basis”, even though their standard practice is “30 day no questions returns”.
    We have pleaded with these folks that the issue should not be us or them, but rather the manufacturer who has such poor QC. These lenses should be returned to Sony and make them pay the costs such practices incur (I guess that is an unlikely result). In any case, they have determined we are not a desirable customer as most folks never complain about “bad copies”.
    The search for a good copy of a $1000 lens that is to be kept for a very long time should not punish the buyer who wants to insure a quality product. Unfortunately, there are no solutions to this dilemma. We have tried keeping a poor copy, sending it back to the factory under warranty. That has proven to be a very poor approach. Seldom does the repair correct a bad copy to a good one.
    We currently have a set of 8 “good copies”. And the thoughts of changing systems, no matter what the benefits of the new tech available, is of very little interest.
    It shouldn’t be like this . . .

    • Half of the problem is that few consumers care, so retailers and manufacturers don’t bother; they’ll just sell your returns to somebody else. You can’t correct a bad copy at the factory; the elements are already set and usually the problem may be slightly off-centered grinding of the glass or mounting of the elements – you can’t fix that easily. But until more people complain, nothing will change. All I can do is raise awareness with the companies I have some sort of relationship with – Zeiss, Olympus and to a lesser extent, Nikon, are so far pretty good about this; when I get a bad copy I’ll inform retailer and manufacturer, and they’re happy to swap out (and let me pick from whatever stock they’ve got) to ensure I find a good one. Bad ones go back to the factory for analysis. Leica, on the other hand…believe every lens they make is perfect. Until the aperture blades fall off.

      As for changing systems – the thought is scary because you’ll have to go through the process all over again. And that isn’t fun, especially if you’ve got nigh on a dozen lenses to replace (like me).

  4. Simon Pi says:

    What do you make of this, Ming? My 50/1,4 aspherical has its infinity mark coming in and past the main focus line indicator on the lens where we are supposed to see the focus distance. Is this one of those mechanical glitches you are talking about, or is it normal?

  5. Reblogged this on dimaswahyudhiantoro and commented:
    Quality control, sample variation and what it means for photographers

  6. Hi, Tom, what kind of flaws did the camera have? Did you just buy the body or did you get a lens, too? If so, how was the lens? Through what channel/outlet did you purchase? Cheers, Nick.

    • Tom Hudgins says:

      Apologies to Ming for using his space for my personal. Hello Nick. Correction: It was an OMD EM-1. I’m a contented EM-5 owner who received a discount offer directly from Olympus for a refurbished OMD EM-1 body only. I made the purchase directly from Olympus. After having used the camera for only four hours it froze three times (had to remove battery to restart); the shutter button triggered prematurely with the slightest pressure; the operation of the dial surrounding the shutter button was stiff as if bent; autofocus was not working properly during controlled tests (was not a problem with the lens); and changes I made to some menu settings would not register. Having used a similar EM-5 for the past year, I knew that this OMD EM-1 camera had both mechanical and electronic problems which I believe should have been detected and fixed during a refurbish process. As per Olympus policy I returned the camera for a full refund which took six weeks. While I have no experience with their refurbished lenses I will not buy anything refurbished from Olympus again. On the other hand, if you are in the market for a manufacturer-refurbished Zeiss lens I had a better-than-good experience with a dealer purchase and subsequent registration with Zeiss. As Ming says (buying factory refurbished) differs from company to company.

  7. Dwaine Dibbly says:

    What about manufacturer refurbished lenses? (I’m thinking specifically of Olympus.) Do those receive some sort of calibration as part of the refurbishment process, or are they just cleaned up and given a shine?

    • No idea, I’d imagine it differs very much from company to company.

    • Tom Hudgins says:

      Dwaine, as I stated earlier, my experience with a refurbished OM-1 body from Olympus was very disappointing. I saw no evidence that the camera had undergone any thorough testing and it was seriously flawed. With that experience I would say that a refurbished lens from Olympus would be a hit-or-miss proposition. I will not buy from them again.

  8. On a practical level Ming it is difficult at best to get more than one lens to test, not impossible though. The Leica Store LA allowed me to sample three 24mm Super Elmar S lenses, a pricey lens. All three showed differences, very easy to detect with simple testing, the best was a refurbished. My next lens purchase will be from them also, I can (at least) test the intended purchase lens against their display sample. The value is avoiding the dogs, and yes Leica dogs exist, that is the benefit of learning mtf curves. The mtf curves once learned and associated with a lenses output gives a great advantage in future lens decisions and purchases.

    My experience with lenses in an attempt to get proper performance from a poor sample is that it can not be done. Very reputable repairman and Leica will fix or repair a lens and this alone may result in better output but if a poor sample is sent a poor sample will be returned when no repair is necessary. Roger Cicala at Lens Rentals in his blog says that a poor lens can be improved to average, but don’t expect much more, they are custom handling each lens for their rental business and as far as I know this kind of service is not available to the general public. Buying a lens is a crap shot, new, used, refurbished, ebay, it doesn’t matter, there is no guarantee it will be a good sample. One last thought, when shooting images unless it is extremely demanding even poor samples can result in beautiful photographs.

    Thanks for a great article, it is fun to get mired in all the technical aspects of photography, and analysis may lead to despair but somehow great photographs can be made despite it all.

    • I’m glad somebody agrees with me – reality is most people lack the access and experience to tell.

      The only other thing is to make such compelling images that technical qualities take a distant back seat. It is of course preferable to have both though…

  9. Carl1981 says:

    I think this blog is the empirical proof that you can make technical good photos with hundreds of cameras/lenses available today. There is no significant difference between the iPhone 5 and the Nikon D800.

    I also love to read gear reviews so my advice is a little bit schizophrenic:

    People go out and make photos with the camera you have at hand instead of reading gear reviews and have no own photos. Or with other words: Life your life instead of reading about the life’s of other people.

    • I’m going to have to qualify your statement: there’s no difference *compositionally* but potentially a truly enormous one depending on your output medium and size.

      • Carl1981 says:

        I think the raw image quality of the digital cameras for the professionals manufactured in 2004 (e.g. Nikon D2H) are not so good as the high-end 2014 consumer cameras. However we have great pictures from this time.

        • Unquestionably. The image is not made by the camera, it’s determined by composition – which is entirely camera-independent.

          • Carl1981 says:

            🙂 Good to see that you digested this conversation in your latest article (“Beyond the numbers: what’s next?).

  10. For readers who want a quantitative look at lens variations and all the things that go into testing them, Roger Cicala at LensRentals.com has been measuring many samples of the same lenses and reporting on his findings: http://blog.lensrentals.com/blog/category/technical-discussion/lensoptics

    Here’s one example on the difference and variations of the Mark I and Mark II Canon 24-70/2.8 EF lens: http://www.lensrentals.com/blog/2012/11/canon-24-70-mk-ii-variation Check out that scatter plot.

    The results are eye-opening. It was his frustration with manufacturers not wanting to fix lenses because they were “in-spec” that lead to his latest venture in constructing a new testing machine (OLAF) that is exquisitely sensitive and illuminating of lens aberrations. Check it out — it’s very interesting stuff.

  11. Martin Fritter says:

    Thought provoking and worrying. One could conclude that no reviews of current state-of-the-art high end photographic gear can be regarded as conclusive. Is it possible that the technology has become so refined that it simply can’t be manufactured and/or maintained reliability? (Some sort of correlate of the 2nd law of thermodynamics?) On a practical level, is a .1 % deviation from specs in the specification-chain of a sota configuration sufficient to vitiate its value? e.g.: Is a 36 mp camera only useful for very specific, highly quality controlled applications whereas a 16 mp one would produce better results more of the time under less controlled circumstances? A great deal of important photographic work is done under conditions where shot discipline is less than ideal – sports, journalism, documentary. Is the search for high-end gear kind of pointless? I mean, why own a Ferrari if you really want to go camping and there aren’t any Ferrari shops nearby anyway?

    In essence, it seems that the returns have have diminished to almost nothing.

    • They’re very small returns indeed for most. However, it’s no different to any other mature discipline – there are specialized tools for the experts, which wealthy amateurs might buy, and others cover. Sadly reality can be very different. The only difference is that these tools are becoming closer and closer to the teach of the average consumer – but education hasn’t caught up yet.

    • Martin makes an excellent point, we are all slaves to consumerism and if like me you grew up in the 80s where owning the best HIFI system was the purpose of your life constantly spending money on new amps decks speakers and cables etc you will understand what a waste of time and money chasing perfection can be
      I distinctly remember upgrading to a D800 and investing in a piece of software to check its focus ability only to get negative results but then as a novice realizing I didn’t have sufficient experience to know 100% that my testing or in fact the software was accurate, I then thought how foolish we all are as consumers spending such amounts of money on products that the manufacturer cant get right out of the box in the false belief that we are improving our photography and life’s in general
      Im sure most of us would do better spending money on learning how to make better and more engaging pictures than pixel peeping.
      It would be great if there were more truly competent photographers around offering to share their skills and classes at a reasonable price where one could then begin to see real improvements in ones output
      I guess 50 years ago the average Joe just didnt have money to spend on fancy gear and so didnt have this concern. Maybe we just have too much disposable time and money today ?

      • Or we’re too susceptible to marketing. If you can spend thousands on gear, it doesn’t seem unreasonable to spend a few hundred on education…

    • Martin

      Depending on what you are making a .1% deviation from the ideal specification can be a huge difference. If we say our base measurement is 1.0 inch (25.4mm) then a .1% deviation is 0.001 inch (0.025mm). Which is a fairly precise tolerance for many common products we use, such as the car engines. Though in industries that have higher standards, such as aerospace, a tolerance of .05% or 0.0005 in may be the norm for critical applications. Now if we consider that in a system of a camera and lens we are trying to adjust wavelengths of light, just about any variation can be significant. the question then becomes how fine or course are our adjustments and how fine can we detect.

  12. rwestcott says:

    Hi Ming. Very thought-provoking. Just surprised that you single out Leica lenses — “hand assembled” — for particular comment. I have 5 Leica lenses and they are all mechanically and optically excellent. In fact I had them scrupulously assessed with the advent of digital and its greater demands on focus accuracy. If there is only a 1 in 5 chance of getting a good lens, based on your experience, then getting 5 good lenses is a 1 in 3125 chance — lucky me. My own 50 lux is without question the best lens I have ever owned.

    What I also find strange is that as a regular follower of the Leica Forum for > 10 years I have NEVER seen a thread about sample variation in Leica lenses. There have been complaints about focus shift and needing to re-calibrate focus alignment (something Leica mostly do at no charge) but no-one has ever raised a complaint about needing to try 5 lenses before getting a ‘good one’. There are complaints about sample variation with other manufacturers. There are plenty of complaints about Leica in one way or another, just not this specific matter of sample variation.

    Do you think your experience was unusual, e.g. were other Leica lenses good first time?

    • rbrooks says:

      I’ve had sample variation problems with Leica lenses and focus, e.g. black vs silver 50mm pre-asph. However, I’m quite critical about fast lenses and close focus. So my variation might be another’s definition of consistency.

      All lenses focus ever so slightly differently from others. My current thinking around this is that it’s probably better to standardise on one lens and learn it’s focus pattern. As my lenses are pre-asph that will mean to learn how it behaves at each aperture and distance — this can take some time and is also quickly forgotten if you put that lens away for awhile.

      • Agreed – I only had one or two lenses I’d regularly use on my Ms for that reason. RF calibration is another thing entirely…

      • I recently bought a Leica 28 f2 cron. It had a back-focusing problem that was quite noticable. I knew it was a definate problem with the lens as my other two Leica lenses focussing perfectly – so it wasnt my camera. I bought the lens from the official Leica store and when I returned to the store with the lens, they were able to check it for me and exchange it for a new one that I had tested extensively whilst in the store. In this case it was fortunate that I bought from a source that had other copies for me to try and were willing to make an exchange.
        I had another problem with a Sigma 17-50mm f2.8 lens for my Sigma SD1M. It too had back-focussing and other focussing problems. Upon returning to the store, they were happy to let me try out three other lenses that they had in stock and let me pick the one I was happiest with for exchange. Although better than the original, the new lens is still far from perfect though :(. (I understand this to be a long-standing general issue with older Sigma lenses)
        The main lesson to be learnt in these two cases is to buy your gear from good camera stores with return or exchange policies.

    • No. I think few Leica owners get to use more than one of any lens, which means they have no basis for comparison. Furthermore, until the M system had live view, it was also very difficult to determine if any loss of resolution was due to the lens, RF or human error.

  13. Vernon S. says:

    Great article. I do wonder which of the camera (and lens) brands do you think generally produces more consistent “good” lenses? I was told that through the rigorous testing at the facility, it was Leica. But this has not been your experience. It would be interesting to get your thoughts on which of these companies produce the best lenses and which produce the worst.

    • Zeiss/ Voigtlander are probably the best off the bunch, followed by Olympus. Nikon and Canon are third. Insufficient experience with the others to say.

      • nehemiah says:

        Having shot several systems, and many lenses, I have been surprised with Sony. The Carl Zeiss lenses can be fabulous (85 1.4, 135 1.8, 24 f2, 55 1.8, RX1), but I have had to return multiple lenses go get good copies. In my experience the pecking order goes: Zeiss, Oly, Sigma (not joking), Fuji, Nikon/Canon/Tamron, then Sony.

        • The Zeiss Sony lenses aren’t made by Zeiss. Nor are the Zeiss Zeiss lenses, either…

          As for Sigma, they’ve suffered from pretty horrible QC in the past, but appear to be improving – now if only the lenses would also hold those calibrations…

  14. I used to design laboratory instruments and high end audio components. Part of good design is good “design for manufacture”. It is possible to make high performance products that are not designed well for manufacture (these tend to be expensive and exhibit wide sample variation) and vice versa. Part of good “design for manufacture” involves reductions in parts counts (where possible) and the addition of physical features on parts that prevent incorrect assembly and aid precise assembly. Another aspect of design for manufacture involves the design of the assembly process, for example, well designed assembly fixtures. The best DESIGNS perform at very high performance levels AND require a minimum of “craftsmanship” to consistently manufacture.

    • Completely agreed – this is however not something we have any real visibility into unless you are brave with a screwdriver! That said, it might explain why MF lenses seem to generally suffer from fewer issues – there are simply less parts.

  15. Great article Ming. One question on adapted lenses causing problems: are you referring to add-on adapters that go between the native lens and the lens mount, or adapted lens mounts (like the leitax mount that Ducloslenses do for instance)?

  16. Adapters don’t really cause any problems in the real world. When they make things off center, they make the entire focal plane off center. That looks really, really bad on a 2D chart, but in the 3D world we actually live in, it just means one side or one corner is in focus slight farther away or slightly closer. A decentered element within a lens can make it impossible to be sharp, but a a decentered entire lens just makes things sharp in a slightly different place. And that rarely matters in a solid, nicely layered composition. It has certainly never bothered all those people using teleconverters on their 500mm f/4’s.

    • It depends on what your threshold of acceptable is – but I’m certainly not going to spend tend f thousands on a good camera and lenses and have the weak part of the chain be a $50 adaptor…

      That said, the absolute tolerance amount relative to flange distance also matters. In practical terms, medium format to medium format isn’t anywhere near as visible as mirrorless to mirrorless.

      • All I’m saying is that a slightly off adapter (or a slightly off flange for that matter) is a different effect the decentering since its shifting the entire focal plane. It’s more like a TS lens, except with the effect so slight (with a good adapter) that it’s mostly invisible in a 3D image and when it is visible it can be accounted for and dealt with using good technique. A truly decentered lens where individual elements are out of align with each other cannot be worked around like that.

        It isn’t about one’s threshold of acceptability unless one’s threshhold involves shooting a lot of brick walls. And I’m certainly not telling you how to spend your money. I’m just wanting to make a distinction. An slightly off adapter is a dramatically different entity than a lens with decentering problems. Surely you can recognize at least that.

        • The adaptor could be decentered (i.e. optical axes don’t align with mount axes, NOT the same as optical decentering of misaligned elements within the lens itself), off plane, tilted, or just too thick/thin. Lots of potential issues, and it appears some confusion in the terminology. You’d need to know what they are (and remember it) before being able to consistently deal with it – I’d rather just buy another adaptor. I am in agreement with you – it is less of an issue on 3D subjects because there’s no flat subject plane anyway – and even less of an issue on long flange distance lenses because the absolute tolerances are significantly less than the focus travel and flange distance. But adaptations of this kind are mostly pointless anyway – MF lenses to smaller sensor mirrorless, for instance.

  17. Tom Hudgins says:

    My nearest camera dealer is a days drive which means I’m generally forced to buy online. As an earlier poster suggested, I’ve tried factory refurbished thusfar with mixed results. I returned a refurbished OMD EM-1 to Olympus which had major issues–it took me six weeks to get my money back. On the other hand, I purchased an excellent refurbished Zeiss lens on EBay. As a bonus, when I registered the lens, Zeiss gave me a 2-year warranty instead their 90-day refurb warranty. Obtaining a ‘best’ copy is a painful process when purchasing online but it can be done. I do envy someone like yourself who has a well-stocked dealer nearby.

    • A lot of the more exotic stuff has to be bought online too, which makes it difficult to evaluate especially if you’re not even living in the same country. I generally try to test one in person before committing – travel helps, I suppose…

  18. Steve jones says:

    Rather like eating in a restaurant in a Foreign country and not knowing if the hygiene in the kitchen is below ‘tolerances’, If we worry about it we won’t eat the food. We have to trust that the food will be good, Same with Leica lenses, we tend to expect that they are as good as they claim to be. If not, what can we do about it? i suppose this applies to everything in life because there will always be imperfections.The problem is, the more money we pay, and the higher the perceived quality of the product, the more we expect that product to be perfect. And as your experience shows we might be disappointed. My Leica 50mm 2.8 Elmar has a focus ring that ‘sticks’ sometimes and can be quite annoying but the sharpness and optical quality is superb so I’ll probably never part with it! Of course, I could have Leica service it but then I’d have to worry they might mess up the focus accuracy on re-assembly! I do think that at the price, and with all the hype, Leica needs to do a better job with quality control. I sure don’t expect to ( and never could afford to) have to buy six lenses to get one good one. I’m hoping the accuracy of the M adapter and M lenses on the Leica T will be accurate enough. But who knows???? I’d guess even the best Swiss watch movements aren’t accurate without adjustment over time?

    • Yes + Whilst appreciating, dazzling depth and brilliant analysis, would to an extend agree with Steve view

    • Don’t buy a mechanical watch if you’re looking for accuracy. As with any mechanical device there are many parts with individual tolerances and these can either cancel out or compound. I agree though: no way to tell unless you try…but if a trend emerges after several attempts, then perhaps it isn’t random.

  19. FredrikRedin says:

    Hi Ming! Nice article as always, made me think about lens wiggle (a bit of play/loosening) between the lens and the camera mount. Do you feel some lens wiggle is always acceptable? How often do you encounter it?

    • If you don’t have any free play the lens won’t be able to mount. But there are tension springs behind the mount flanges that should negate it; if it’s loose – ie doesn’t require a firm tug to move – then send it back.

  20. Reblogged this on Giai01's Blog and commented:

  21. I think that Leica also, not only Zeiss, produces its MTF graphs with real measures instead of computer simulations.

    • I was told they were theoretical. And judging from the spread of lens quality I’ve seen for the same lens, they’re definitely NOT measured.

      • Ming

        I want to say that I have read in the past the the MTF curves posted by by Leica are the actual resulting curves from testing a real lens. More than likely this would be a prototype or a sample from the initial production lot. If this is true, we have no way of knowing if this is an ideal case, the best sample, or average sample. Though, given it was published, I would it to be a best sample.

        In my time in photography I have only heard of one company providing an actual MTF chart for each lens sold. This company was Sinar, and the story said they would provide the chart after the newly purchased lens was registered. I have never purchased new Sinar branded lens, so I haven’t verified this claim personally. Though the few Sinar, and Linhof, branded large format lenses I have owned have all been excellent.

  22. Linden says:

    Do manufacturers re-check samples and calibrations between body and lens samples if you (the customer/ owner) submit them for this checking? Can a “bad copy” be turned “good” by doing so?

  23. Yeah, the guys who responded in the thread talked about the economic incentive for the manufacturer to package kits with equally good copies to the separately packaged lenses.

    I would be interested to know just how much the Olympus PRO lenses get tested. If, after testing my purchased copy, I am not happy with it should I ask for an exchange? Or, is that something you recommend each buyer works out with their reseller? I take it that you don’t generally buy lenses online, sight unseen (untested)? Cheers

    • I almost never buy lenses online, or if I do, I’ll buy them from a place that will support exchanges/ returns – often there are lenses that I want that I simply cannot get locally due to supply or them being vintage. That said, if I’m not happy with a NEW lens, I don’t care who lands up doing the exchange/ replacement – you’re simply not getting what you paid for; why should you accept that?

  24. Such an excellent article, Ming. For two years, I’ve tolerated back-focused lenses because I assumed that they were performing as promised. Only in the past few weeks did I discover strong back-focusing and corrected the lenses. Now, they’re like they should have been in the first place. It’s frustrating to be sure. But had I thought this through and tried more lenses in the first place, I may have recognized the tolerance issue and corrected it immediately. We’re good to go now. 🙂

    • And it makes quite a difference, doesn’t it? The problem with any manual focus camera is that you’re never quite sure if it’s your eyes or some miscalibration somewhere…

  25. Very timely, Ming. I was just having a discussion about whether or not a manufacturer might package “the lesser instances” of lenses into their kits. For example, I’m looking to by the E-M1 with 12-40 f2.8 PRO as a kit and I wondered if I should be purchasing the lens separately. Just in case they “bin” their copies differently as they come off the assembly line and keep the better lenses for those customers paying full price for the separately packaged and sold item.

    But, as you say, probably not anything to be concerned about even if they did, given the lower resolution of the m43 sensor. Cheers.

    • I doubt it – that would imply several things: firstly, they bother testing them all to that extent, and secondly, you get punished for buying more stuff (especially in the instance of the E-M1 kit).

  26. Adding on to your point about reviewers not actually looking at this difference, it’s common practice for brands to actually provided a ‘media tuned’ version of the mass production models for testing to reviewers. They’re very strict on indicating pre-production but when it comes to these ‘tuned’ versions, because they’re based on the final firmware, etc, it doesn’t make it a legitimate basis either because it isn’t what everybody else in the market is actually buying. Media sets are common practice and oftentimes the difference can be quite dramatic (in my personal experience). The only way to really test the validity of a brand’s claims is to test out the products purchased over the counter (even for camera bodies) and as you say, a lot of them to really tell

    • This may well apply for hardware calibration, but it wouldn’t make sense for software calibration due to the amount of work required. Interesting you should mention this especially given your previous insider knowledge…

      But sample variation – deliberate or not – is why I inevitably land up testing several copies of whatever it is to check that the results I’m seeing are actually consistent. One more reason why reviews and testing are a bit of a chore at times…

  27. Just to throw another concept into this, there are times when buying refurbished makes more sense than buying new. Obviously this depends upon the quality of refurbishing, and the nature of the original problem with the gear. The thought behind this concept is that additional inspections lower the incidence of problems, though like anything else in life, results may not be the same for all end users. 😉

  28. isaacimage says:

    Once again -great article – Thank you !


  1. […] good and bad examples for each lens individually. See for example this recent article about that: Quality control, sample variation and what it means for photographers ? Ming Thein | Photographer See also reviews of kit lenses in the net: Nikkor AF-S DX 18-55mm f/3.5-5.6G ED II – Review / Test […]

%d bloggers like this: