How is this different from using a small aperture size?
When you reduce aperture size the depth of field increases. So for example when you use f/16 pretty much everything from a few feet to infinity is in focus.
Is that actually true? I do astrophotography through an f/10 telescope and its focus is very sensitive. I use a focuser that moves the camera 0.04 microns per step.
Not doubting you, just asking to understand. Astrophotography doesn't always behave the same as terrestrial photography
in addition to aperture, percieved depth of field greatly depends on:
- focal length (wider is deeper)
- crop factor (higher is deeper)
- subject distance (farther is deeper)
compared to your telescope, any terrestrial photography is likely at the opposite extremes, and at a disadvantage everywhere but subject distance.
but, focus is most mechanically sensitive near infinity. adjustment creates an asymptotically larger change in the focal plane as infinity is approached.
in a point-and-shoot camera with a wide lens at f16, "infinity" basically means across the street.
I believe the lytro camera was a plenoptic, or light field, camera. Light field cameras capture information about the intensity together with the direction of light emanating from a scene. Conventional cameras record only light intensity at various wavelengths.
While conventional cameras capture a single high-resolution focal plane and light field cameras sacrifice resolution to "re-focus" via software after the fact, the CMU Split-Lohmann camera provides a middle ground, using an adaptive computational lens to physically focus every part of the image independently. This allows it to capture a "deep-focus" image where objects at multiple distances are sharp simultaneously, maintaining the high resolution of a conventional camera while achieving the depth flexibility of a light field camera without the blur or data loss.
Something I find interesting is that while holograms and the CMU camera both manipulate the "phase" of light, they do so for opposite reasons: a hologram records phase to recreate a 3D volume, whereas the CMU camera modulates phase to fix a 2D image.
I remember Lytro. There was a lot of fanfare behind that company and then they fizzled. They had a lauded CEO/founder and their website demonstrated clearly how the post-focus worked. It felt like they were going to be the next camera revolution. Their rise and demise story would make a good Isaacson-style documentary.
If I recall correctly they got scooped up by Google and their team was merged into various Google teams. I was disappointed to hear of their fizzling as well. They were just starting to dive into serious movie production light field cameras when it happened. They had an incredible tech demo on their website showcasing its power. I can't seem to locate the original but there are bits of it in the linked video.
I think the product was just too early for its time, and there is not much demand for it.
For what it's worth, the founder (Ren Ng) went back to academia, and was highly instrumental in computer vision research, e.g. being the PI on the paper for NeRF: (https://dl.acm.org/doi/abs/10.1145/3503250)
I don't think it was quite too early, it just makes tradeoffs that are undesirable.
Lytro as I understand it, trades a huge amount of resolution for the focusing capability. Some ridiculous amount, like the user gets to see just 1/8th of the pixels on the sensor.
In a way, I'd say rather than too early it was too late. Because autofocus was already quite good and getting better. You don't need to sacrifice all that resolution when you can just have good AF to start with. Refocusing in post is a very rare need if you got the focus right initially.
And time has only made that even worse. Modern autofocus is darn near magic, and people love their high resolution photos.
I find it very useful for wildlife photos. Autofocus never seems to work well for me on e.g. birds in flight.
It's also possible to generate a depth map from a single shot, to use as a starting point for a 3D model.
They're pretty neat cameras. The relatively low output resolution is the main downside. They would also have greatly benefited from consulting with more photographers on the UI of the hardware and software. There's way too much dependency on using the touchscreen instead of dedicated physical controls.
I'd argue the opposite, consumers need more resolution than pros.
A pro will show up with a 300mm f/2.8, a tripod, a camera with good AF and high ISO, and the skills, plan and patience to catch birds in flight.
But all that stuff is expensive. The consumer way to approximate the lack of a good lens is a small, high res sensor. That only works in bright light, but you can get good results with affordable equipment in the right conditions. Greatly reducing the resolution is far from optimal when you can't have a big fancy lens to compensate.
And where is focus the hardest? Mostly where you want to have high detail. Wildlife, macro, sports.
You couldn’t offload and use the refocus feature without their software and 4MP was half what smartphones were doing in 2012 (rapidly increasing above 8MP after that). Fixed lens so you couldn’t improve the image quality with glass - that’s totally understandable given the product but it is still a limitation to the visual quality.
That’s a bad recipe for casual and professional users alike. Can’t ingest into your workflow quickly, images are low res, can’t improve the image, and your smartphone was better just missing one, admittedly neat, feature. If that existed in phones people would use it like crazy I imagine.
Too narrow of a use case IMO, too many compromises for one feature, hence why it failed.
I doubt resolution is a limiting factor LCoS (transmissive ones are lower resolution typically though, but you could build reflective) phase modulators are available in 4k resolution (and maybe even higher). And I don't know if you need that much resolution because the regions you're trying to focus are quite broad, in fact I suspect the resolution of your phase modulator would not limit resolution but the max distance between focused regions, because it would set the max phase ramp you can achieve.
Loss, i.e. equivalent aperture is a different matter and I think this would imply quite a light loss.
I looked into the details at the time, and as I recall the camera had lower resolution than the 4 MP of the sensor because of the microlens array. A lot was written about this at the time.
I remember a friend, who was a photography buff, was quite excited about the camera. But he didn't actually buy one.
The paper describes a split Alvarez (Lohmann) lens [1,2] with a phase modulator between them. I didn't do the math, but it looks like the phase modulator is optically equivalent to a mechanical shift of the Alvarez lenses over regions of the field of view. Alvarez lenses have higher aberrations, and are relatively bulky, compared to normal lenses. AR was referenced in the paper, but this lens will be hard to make compact, and have great image quality, over large fields of view.
If your samples are fixed, you can take a z-stack spanning the entire area you want to capture and then use max intensity projection to collapse them all into one clear image.
But yeah this new camera would be good for living microbes.
It is a new neat idea to selectively adjust focus distance for different regions of the scene!
- processing: while there is no post processing, it needs scene depth information which requires pre computation, segmentation and depth estimation. Not a one-shot technique and quality depends on computational depth estimates being good
- no free lunch. The optical setup needs to trade in some light for this cool effect to work. Apart from the limitations of the prototype, how much loss is expected in theory?
How does this compare to a regular camera setup with lower aperture? F/36 seems excessive for comparison.
- resolution - what resolutions have been achieved? (maybe not the 12 MPixels of the sensor? For practical or theoretical reasons? ) What depth range can the prototype capture? "photo of Paris Arc de triumphe displayed on a screen". This is suspiciously omitted
- how does the bokeh look like when out of focus? At the edge of an object?
The introduction of weird or unnatural artifacts would seriously limit the acceptance
Don't get me wrong - nice technique! But to my liking the paper is omitting fundamental properties
While this methods has no post processing, it requires a pre processing step to pre-calture the scene, segment it, estimate depth an compute the depth map.
Ricoh has some nice EDoF cameras that would make for great QR scanners. They do it the classical way, though. That is, distance dependent chromatic aberration.
In the industrial context you could use a telecentric lens, but it necessarily needs to have large input aperture to have a reasonable field of view.
When you reduce aperture size the depth of field increases. So for example when you use f/16 pretty much everything from a few feet to infinity is in focus.
reply