tellure

Yeah this is definitely the future of stabilization, it's been working great on the GoPro's (Hypersmooth) for a couple years now.  I'd also love if we could shoot in slightly higher resolution so the stabilized & cropped image was still 4K.

5K anyone?

The A7S3 has lots to like but I'm not sure if I'll be upgrading just yet.  Personally I'm hoping we get a 24MP A74 that does 4K 60 (with 6K downsampling for more sharpness) and allows for the 1.5x APSC crop at full 4K.  I use that APSC punch-in a lot on my A9 and would easily give up 120fps for it (60fps is still 2.5x slow-mo at 24fps playback which is pretty good for most purposes).

Kind of bummed the color is not sorted yet.. but maybe EOS HD Pro Color can improve things.

Jeeze, still no 4k60p.  Anyone know if they at least added picture profiles this time?

I just subscribed to HBO through Amazon to see what the quality difference was like.  I went back and watched a bit of the much-maligned Episode 3 and it looked way, way better.  HBO through Amazon looks like true 1080p at a good bitrate (for streaming).  HBO Go / Now still looks like upscaled 720p with terrible compression artifacts.  This is watching from Los Angeles, BTW.

21 hours ago, BasiliskFilm said:

You are turning a full frame camera into a crop frame camera - the centre 50% of the sensor is about the same size as a micro 4/3 sensor. The standard way of working out full-frame-equivalence is to multiply the focal length and aperture by the crop factor, in this case 2x. The actual focal length and aperture remain the same, obviously.

@BasiliskFilm That would be true if we were only using the center 50% of the sensor here.  But in this proposed feature we are still capturing an 8K image from the entire sensor, and cropping it during the encoding process.  So the image circle projected on the sensor and sensor size remain the same.  It's literally the same as taking a full-frame 8K photo and then cropping it in post, which I'm guessing you'd agree wouldn't require any multiplication of aperture because the depth of field is already baked into the original full-frame 8K photo.

I shot a few tests last weekend on the A9 to test object tracking in video with the new firmware and it was really impressive.  I was shooting my wife at ~35mm f/2.8 from a few feet away up to less than half a foot, so fairly shallow DOF.  She was moving forward and back, left and right, and the tracking was keeping her almost perfectly in focus the whole time.  Only when she moved almost out of the frame did it start to lose her, but regained focus in less than a second.

I'm excited to try using this in some more real-world situations like on a gimbal, but so far it looks to be what I've always wanted - awesomely accurate tracking AF with touch-focus.  Huge kudos to Sony for adding this in a firmware update. Now if they would only add picture profiles so we could get an EOS HD color profile on the A9...

@Mokara I agree digital zoom always compromises image quality, but we're not talking about a digital zoom here, which takes a fixed sensor readout and up-res's the image via interpolation.  We're talking about cropping an 8K sensor at variable sizes until you get down to 4K at the maximum crop (2x zoom).  So the final readout at maximum zoom would still be native 4K.

It's possible that downsampling at all the varying sizes in between 8K and 4K could introduce artifacts though.. just like downsampling an image in photoshop by 46%, for example, is never as clean as 50% or any multiples of 2.

@BasiliskFilm I had that same thought and I'm happy to be proven wrong here since I sat out the aperture equivalence wars but I don't think it applies here.  As I understand it, aperture equivalence should be calculated when comparing sensors of different sizes with equivalent framing.  But here we're talking about one sensor size - 8K - just cropped down to varying sizes and downsampled to 4K for the final output.

Think of it this way - imagine you took a series of 8K photos of a fencepost at 50mm f/1.4 and stitched them together into a time-lapse video.  There's lots of creamy bokeh in the background since you're shooting at f/1.4.  Because the final output of your video is 4K you have lots of room to crop in post.  So you add some zooms in your thrilling fencepost video to get more detail by cropping the image.  In the final seconds of the video you end at a 2x zoom, and now the *framing* (and only the framing) of the fencepost photo is the same as it would have looked like with a 100mm lens.  All that creamy f/1.4 bokeh in the original shot is completely unchanged since the contents of the original 8K photo are unchanged, it's just been cropped.

Unlike regular aperture equivalence calculations where the image being compared is being projected on different sensor sizes, in this (wishlist) feature the image projection is always over the whole 8K sensor.  Internally the camera would be cropping and downsampling the image before encoding it into the 4K video stream.  Hope that makese sense, but again happy to be proven wrong, I'm not an expert on the hotly debated topic of equivalence.

Cool, thanks for the info!  That JVC feature is exactly what I'm after.  Now they just need to get it in an 8K sensor outputting to 4K, as opposed to a 4K sensor outputting to 1080p :).  Still really promising that it's been done.  Makes me hopeful it'll eventually make its way to the 8K cams.  And they call it "prime zoom" no less.  The appeal of zooming with a prime seems pretty strong. 

A lot of us don't need more resolution at this point.  The idea of 8K sensors seems appealing not for resolution but for other potential features, like even better low light performance, as Andrew recently wrote about.

But what if your 8K sensor could zoom (by cropping) up to 2x, without any loss of quality, while outputting a 4K signal?  Right now many of us love that we can punch into APS-C from full-frame for an extra 1.5x zoom while still getting native 4K.  An 8K sensor could allow us to get to 2x while still getting full 4K resolution at the long end of the zoom.

Your expensive 24mm 1.4 prime could become a 24-48mm 1.4 (a lens which doesn't exist in full frame).  But to make this a really killer feature the zoom needs to be realtime.  Right now we have to stop recording to switch between full-frame and APS-C.  What if you could zoom while recording?  What if you could zoom in small increments, and control that zoom using any re-mappable dial on your camera, or even use your manual focus ring if you were using AF-C?

But wait, why not just shoot everything at 8K and then crop in post?  Composition, of course.  Being able to make those framing decisions in realtime and see it on the screen would be great, just like having a native zoom lens allows for more composition choices on the fly.  And of course you wouldn't need to store those larger 8K files.

Taking the functionality even further, if 8K sensors could do this cropping in realtime then it could even allow for a power-zoom feature - e.g. the ability to set a zoom rate and then do a slow crawl-in or even a smash-zoom in/out.

Sony's Clear Image Zoom is the closest thing we have to this right now, but of course it's not cropping, it's doing interpolation to give us an up-res'ed image from the same sensor area.  It's also sorely lacking in usability (can't assign it to any wheels or dials, and it only zooms in stepped 0.1x increments).  Worse yet, all the focus functions are disabled when CIZ active, so you lose the ability to set a specific focus point, change the focus point, or use object tracking.  Which is a shame because the quality is actually pretty decent, as shown below.

I imagine the processing horsepower needed to do a realtime crop smoothly across an 8K sensor is probably pretty huge, and I don't expect this kind of feature in the first or maybe even the second generation of 8K sensors but it seems like something that could happen eventually. Given how much I use the 1.5x APS-C crop now I would use this feature a ton.

Since the environment seems like the biggest determining factor for getting great shots here (shooting a fast-moving subject in a natural environment) I would suggest focusing on cameras and gear that work best for those kinds of shoots, like Snowbro recommended.

BTM_pix posted this info about the Wiral LITE wire rig that looks like it would be great for mountain biking:

I recently bought a GoPro Hero 7 black and would definitely recommend it for the new hypersmooth stabilization (check out the many YouTube vids / comparisons) if you're doing any POV stuff or ride-along shots.

11 minutes ago, webrunner5 said:

I don't know a lot about HDR in video. But I am sure it has to be 10 bit or better and have a Log profile to do it. So no clue what per-pixel ADC helps on that part for HDR?

 

Just theorizing here since I don't know a ton about sensor design but if you have a per-pixel ADC then could you, for example, expose for the blown-out windows in an indoor shot and then boost the gain only on the darker pixels to get the equivalent of a bracketed HDR shot?

36 minutes ago, Eric Calabros said:

You get lower DR at high ISO cause the amplification needed to increase the brightness of darker areas is too much for already bright areas. With this new pixel design, it could be possible to amplify only those pixels that need to be amplified. So you get near ISO 100 DR with very low noise at say ISO 12800.

Other than fabrication complexities, there is big problem in profiling the output, because it's not going to be linear, I guess. 

Ah, thanks.  That does sound pretty awesome.

10 hours ago, androidlad said:

One ADC per pixel is called Electronic Global Shutter.

For us laypersons can you elaborate on what benefits this brings?  Does this get us per-pixel HDR?

19 hours ago, IronFilm said:

Already exists for sub $2K!

The Z Cam E2

Uhh yeah but that's a Micro-4/3rds sensor.  Big difference to doing 4K 120p in full frame, which is what I was predicting within 5 years.  Sorry that wasn't clear.

On 1/28/2019 at 6:27 PM, kye said:

I remembered another possible use for 360 cameras.

360 cameras + 8K (or higher)
360 cameras might make a great second angle for professionals who shoot events or weddings, especially if you put them on an extendable pole and put them up high.  I think they'd make great wide shots that would look like static drone shots but could be gotten in places you can't feasibly use a drone.  Eg, a wide shot of the crowd at a concert (either looking towards the stage, or back from it) or at the back of a wedding ceremony for a wide view of the bride walking down the aisle, etc.  This wouldn't be too hard to use either, just mount the camera on an extendable arm and mount it to your tripod (or a separate one), then whenever you setup your camera, hit record on it, extend the arm, forget about it and get your normal shots, then retract the arm, hit stop, and deal with the rest in post.  You don't even have to watch the footage it gets, just knowing what shots you want from it and scrubbing back and forth to find a good snippet is enough.

This is a pretty interesting concept.. a 360 cam on a stand / pole that you can throw up in a few locations to get lots of coverage does sound pretty good for events.  What if you could mount it on a roomba-style robot base to get some motion in those shots? :).  That Insta360 Titan already has WIFI video preview, seems like it would be pretty great for events to be able to control several of these remotely. 

With respect to cropping into the 360 cam footage, is there any issue cropping into the side of the frame, wish all that distortion from the fisheye lens? 

5 hours ago, kye said:

Electronic variable ND + better ISO performance (or dual ISO, or Triple ISO?)
This combination could create a camera where we control the creative parameters (SS, F-stop) and the camera manually exposes using only non-creative controls (ND, ISO).  This means you can set whatever aperture you want, 180 degree shutter, and then take exposure off your plate and think about more creative aspects of shooting.  The "cinematic vlog" crowd might help with this as there's a ton of them and they care about SS and F-stop but want fast-paced shooting, and will be adding their voice to the pros who also want built-in NDs.

NLE + AI face-tracking and shot analysis
Not something people talk about, but there might be benefits to NLEs being able to analyse your footage.  Immediate benefits would be logging footage by reading slates (if you have them), detecting takes (visually similar shots repeated over and over), tagging of shots with which faces appear in them, etc.  Combined with smart bins this would save a lot of leg-work, especially in doc work where you have huge amounts of footage.
For the less serious users (some people felt that FCPX was really iMovie, so Apple are trying to reach these people) you could also have expression detection, framing detection, and other ways to rate clips.  The existing 'choose the best shots and make a sequence' technology could be incorporated, creating a rough-cut with a single button.  More advanced features would be to watch you through the webcam as you view shots and further refine based upon your reactions.  
This could be huge as people are recording video and photos all the time, and editing your photos and choosing the best ones has been made easy but video editing is still something the average person doesn't even attempt.

Your description of the automatic exposure control using built-in variable ND is exactly how the Auto ND feature on the Sony FS5 works as far as I know (video).  I would definitely love to have that feature in a smaller body.  Fiddling with screw-on variable ND's to get the shutter speed where I want it is always a hassle and just takes time away from the creative aspects.

I agree with you about the face tracking and shot analysis features to help mainstream consumers get into editing.  In the same way that AI neural nets are taught what type of content to look for in image processing you could probably teach them what kind of shot types are attractive to the YouTube audience and then have it use that to rate a user's raw footage, then assemble a rough cut like you suggest.

Let's hear your ideas, people!  Some of these might be things you expect to happen in the next few years, or maybe as far out as the next 10 years.

Personally I am super stoked for the next 5-10 years in camera video tech.  We're all used to improvements on the hardware side and that will continue of course, but with more and more image processing advancements happening in software I predict we'll start seeing a lot more features coming from advanced processing techniques combined with new hardware, like the GoPro 7 hypersmooth.

Ok let's get started.  I've chunked up my predictions into some major categories but feel free to add your own if they don't fit into these:

AutoFocus:

There's been a ton of innovation in AutoFocus over the last 5 years and I think we'll continue to see more, including more stills AF features making it to video, like runtime eye-AF which is now shipping on the Sony A6400 and upcoming A9 firmware update.

• Face registration with priority
  • This is another existing stills feature that we don't have in video yet (to my knowledge)
• 3D Focus tracking?
  • Generating 3D models from multiple high resolution stills (photogrammetry) is tech that's in common use right now in a variety of fields (e.g. Google Earth)
  • It's reasonable to imagine that with enough processing horsepower and good software a camera could build a 3D AutoFocus model of a subject (rotate around subject to get data)
  • With a 3D AutoFocus model it's possible we could have highly accurate tracking even as a subject moves / rotates in space

Sensor / Resolution

I'm not super well versed in sensor tech so below are some of my layman's predictions.  @androidlad, wanna chime in here?

• Realtime HDR?
  • High-res quad bayer sensor which combines short and long-exposure pixels into an HDR frame.  This is a feature on recent smartphones (see EOSHD thread)
  • Sony's STARVIS Micro-4/3rds sensor does quad bayer HDR already for video
• Full-readout 8K with 4x Clear Image zoom?
  • The Sony A9 can already do 2x Clear Image Zoom without any noticeable loss in quality (example)
  • With a full readout of an 8K sensor (33.2MP) downscaled to 4K, we could get 4x Clear Image zoom while still outputting a full-res 4K signal
  • Imagine being able to use, say, the 24mm 1.4 GM as a 24-96mm 1.4 zoom lens.  Sounds pretty sweet to me

Stabilization

• In-camera image-processed stabilization  (a la GoPro 7 hypersmooth)
  • I've got a GoPro 7 and this stabilization tech truly lives up to its name.  All done in-processor in realtime in 4K right now (albeit with a much smaller sensor readout). Bringing this tech to larger-sensor cams seems like a big win

Codecs / Compression

• 10bit 4:2:2 / higher bitrates / blah blah
  • Already here in the GH5, only a matter of time before it trickles down to more cams
• Lossless compressed RAW (a la BlackMagic RAW)
  • I could see some other manufacturers making their own version of this type of codec, and maybe selling it as a paid firmware update on prosumer cams for those users that want RAW
• 120p 4K
  • 4K 60p is already here on APS-C, 120p seems reasonable in 5+ year's time with enough processing power / heat management

Realtime encoding features

• Realtime hyperlapse
  • Already on smartphones and the GoPro (which works great - example)
• Simulated Depth of Field
  • This is already a stills feature on modern smartphones and it works pretty well (Google's Pixel portrait mode explained).
  • Not super exciting for those of us that like real bokeh from fast lenses on large sensors but I could see some smaller action cams / vlogger cams making good use of this.

Ergonomics / Body / Usability

• Smartphone touchscreen performance
  • It's a travesty that we still don't have this level of performance in our camera touchscreens given how long smartphones have been doing this well
• Smartphone-style touchscreen interface / usability
  • Again a travesty that $3500 cameras are light years behind a $200 smartphone UI.  Leave the legacy UI mode for people who want familiarity but give us a proper 21st century UI.
• In-camera ND
  • This already exists on higher end models like the Sony FS5.  Obviously this is a much larger body with an APS-C sensor and I'm sure the current system wouldn't fit into an A7/A9 body, but maybe we'll see some miniaturization of this or some kind of on-sensor ND tech.
• Clear Image Zoom on focus ring
  • Ok this is a small idea but imagine being able to swap the functionality of your focus ring (since many of us are using AutoFocus) into a Clear Image Zoom ring, so you can zoom smoothly while recording.  Could even give it a power-zoom mode where you twist to zoom in/out at various speeds.

Connectivity / Net features

• Realtime bluetooth video feed and camera control over for all settings over Smartphone
  • This already exists to a large degree on the GoPro.  The live video feed and changing of camera settings works well although they can't output the video feed while recording.

What are your predictions?

On 1/20/2019 at 2:20 PM, Andrew Reid said:

The A9 does at least get other colour tweaks and Night picture profile on -1 contrast, -3 sharpness is pretty good, I find.

Funnily enough I've been using my Super 16 c-mount lenses on the A9 in 2x crop mode. Maintains crispy 4K res at 2x crop in video mode and some c-mount lenses 75mm and longer cover Super 35mm mode as well!

2x crop with the Clear Image Zoom to maintain 4K I take it?  Do you find it has any additional jitter / image stabilization issues?  I always seem to see a bit of bounce in the frame in the clear image zoom tests:

I must say I've been remiss in not trying it out, I should do some testing with it.  I've been thinking about the potential of switching to the 16-35mm 2.8 as a single all-purpose lens when you can effectively turn it into a 16-70mm 2.8 with Clear Image Zoom.

Looking forward to touch screen tracking.  Still no picture profiles though?  Sigh..

Still hoping they do a firmware update to add picture profiles to the A9.  Then we could get EOSHD color on it :).  Every time I see a new firmware release for the A9 (new one yesterday!) I get my hopes up, but every time it's not in there.  Pretty certain we're never going to get them.

Awesome series!  Camera fumble portrait is such a great tip, can't wait to use it.

Looks really good. Can't wait til this tech makes it to full frame prosumer cams.

Great test, Max's stuff is awesome.  Did anyone else see the color moire on the Pocket 4K video though?  I kept expecting him to call that out but he never mentioned it.  Seemed pretty obvious on those 400% crops on wires and such.  Didn't notice it at all without the crop though (watching on a 4K monitor fullscreen).

@mojo43, I'd also love to hear your tips on crafting the audio tracks.  Your videos have a ton of great sound effects to sell the transitions and action, which makes a huge difference.  Be curious to hear where you get your effects, how you choose and edit them, etc.

Great tips and you're travel videos are very professional.  Would love to hear the video on how to get great portraits, although I feel that like photography it'll largely be about personality - how you can connect and make a rapport with people in a short amount of time.  Something many of us gear-heads are terrible at..

@Mokara, do you notice any speed improvements when editing H264 content?  I saw a mention of performance improvements on their blog last month about the next CC update.  I'd sure love to be able to edit X-AVCS h264 natively and not have to make proxies for editing speed..

Performance and format support

Hardware-based encoding and decoding for H264 and HEVC provide better performance and faster rendering. Improved image processing provides more responsive playback, rendering, and Lumetri Color performance.

https://theblog.adobe.com/creative-editorial-with-faster-workflows/?trackingid=71KW6ZN6&mv=email