Jump to content

horshack

Members
  • Posts

    167
  • Joined

  • Last visited

Posts posted by horshack

  1. 5 minutes ago, docmoore said:

    It is really a good program ... I have had it for close to 15 years ... easily justified the cost in my case.

    And it is great to see that the files are on the card 🙂

    It's definitely a start. The file sizes are off though - it shows only 96K for the MP4 and put the remainder of the card's capacity in bucket labeled .mhk. Not sure if the extraction would successfully reconstruct the file.

  2. 21 minutes ago, docmoore said:

    Might give IsoBuster a try ... I have recovered a ton of stuff when the FAT was completely corrupted ... mainly from DVD/CDs

    It will dump any information it finds and is usually able to tell the type of file from the structure.

    Good call @docmoore. Just downloaded IsoBuster and it found the .mp4. It wont let me attempt to extract it without buying a license.  I'll search out a few other free/open-source solutions first before biting the bullet on the purchase.

    Here's a screenshot of the recovery:

    i-T7jmV8V.png

  3. I looked into what it might take to recover the lost/missing video file after a battery pull. I reproduced the R5 hack scenario on my Canon RP by recording a minute of video on a freshly-formatted SD card and then pulling the battery while the door-sensor was inhibited. Like what Andrew discovered on his R5 the file is not corrupt but actually completely missing, both when attempting to play the video on the camera and when mounting the card on a computer. I then tried running SanDisk's RescuePro Deluxe and after scanning the entire card it didn't find the video.

    Here's what I think is happening. Obviously Canon has to write the video data to the card while recording, since it has no other place to retain it beyond the internal SDRAM buffer capacity. You can see easy evidence of this by watching the card access light continuously flicker. The strategy Canon is likely employing is to defer the writing of the official FAT metadata until the recording is orderly stopped. This would include elements such as the FAT directory entry, which anchors the file, and possibly also the linked FAT allocation tables. This would explain why RescuePro couldn't find the file, since there isn't enough or any recognizable metadata to reconstruct the orphaned data sectors associated with the video. Canon is either caching that information in memory until the file is closed or is writing it somewhere on the card that isn't linked to the official FAT structures. The benefit of this strategy, assuming they're using it, is that it prevents any potential filesystem-level corruption from incomplete metadata updates, since none of the official metadata structures are updated to link to the file until it's closed. The downside of this strategy is that it orphans all the data from the file and makes recovery more complicated.

    There are a few strategies in devising a recovery app for this situation. The first would be to reverse-engineer exactly what if any metadata Canon is writing during the recording and to where. If that orphaned metadata is in the same format as actual FAT structures then it should be relatively straightforward to create a placeholder directory entry and link to it. If the format of that metadata is proprietary to Canon then the process of reverse-engineering its structure would be much more complex. This would all be done by using a sector editor and block-based search tools to compare a freshly-formatted card to one which has a missing video after recording. And lots of effort.

    I was actually an embedded firmware storage engineer for most of my professional career so this would be in my wheelhouse. I'll see how much demand there might be for this before I spent any serious amount of time on it.

    In the meantime I'll also try some other recovery apps to see if they have better luck...in case SanDisk's utility is not doing the job it's supposed to. I recommend others try the same so we don't unnecessarily reinvent the wheel 🙂

  4. 1 hour ago, Andrew Reid said:

    Updated article with more clarifications.

    It is battery backed RAM not NVRAM EEPROM (that's why the Baidu internal battery pull worked) but the basic workings are the same

    https://www.eoshd.com/news/canon-eos-r5-so-called-overheat-timer-defeated-by-a-single-screw-in-battery-door/

    A lot to get our heads around, so please if you are commenting on this topic, make sure to read the full article first.

    Good update. FYI, NVRAM refers to any memory that doesn't require the embedded device's primary power source. This can include actual non-volatile memory like EEPROM but is more typically the battery-backed variety, like the Dallas parts that also include an RTC (real-time clock). Here's a sample Dallas datasheet for the latter variety and excerpt:

    "The DS14285/DS14287 Real Time Clock with NVRAM Control provides the industry standard DS1287 clock function with the additional feature of providing nonvolatile control for an external SRAM. "

    "The DS14285/DS14287 uses its backup energy source and battery-backup controller to make a standard CMOS static RAM nonvolatile during power-fail conditions. During power fail, the DS14285/DS14287 automatically write-protects the external SRAM and provides a VCC output sourced from its internal battery. "

  5. Andrew, the video that won't play-back in the camera after removing the battery (without stopping the recording first) is likely recoverable by reconstructing the missing tail data of the MP4 container. I've never had to do this myself but in checking online there are a few open-source solutions, some of which require a dummy "good file" taken anytime from the same camera that they then craft the tail data onto the truncated file.

    If that solution works then a shooter using the battery hack could just shoot an extra 10-20 seconds of unneeded outro footage that would be tossed from the battery pull. 

  6. 5 minutes ago, Andrew Reid said:

    How can it "overheat" just switched on in the Wifi menu, but do 4K HQ via HDMI just fine?

    The processing loads surely are not higher in the Wifi menu compared to outputting 8K to the DIGIC X image processor, processing it, resampling it to 4K 10bit and sending it out to an HDMI recorder?

    So either the HDMI mode bypasses the cripple clock accidentally (a bug in the cripple hammer). or Atomos persuaded them to turn the timer off?

    Nothing else makes sense.

    By overheat do you mean shutdown or do you mean presenting the overheating warning to the user? 4KHQ seems to run over HDMI while the overheat warning is active (per my understanding of mechanicalEYE's post above, so it's not clear what unique significance that warning carries since the camera still operates externally when its active.

  7. I actually wanted to use Horseshit as my online handle around the web but it was either already taken or disallowed by profanity filters 😀

    I would speculate it's the combination of the sensor and DIGIC as the main heat sources, in combination with the supporting DDR. The camera can record unlimited 4KLQ, which involves line-skipping/binning instead of oversampling, which reduces the workload for both the sensor (less data sampled per frame) and for DIGIC (less data to process coming off the sensor). The camera can also record long-run times using 4KHQ externally, which means the internal recording path is not used, which means less data flowing through DIGIC for processing for compression. If we presume there's a total operating heat budget and each of these components can contribute to exceed it, it makes sense that there's a matrix of settings (resolution, oversampling vs LQ, compression vs not) which allow the camera to record longer in some combinations and not in others. I'd also speculate the thermal algorithm uses both an instantaneous sampling of current temperature and also a projection of the thermal ramp expected based on the settings used, which in combination result in the projected available recording time that's reported to the user and also used as the thermal shutdown trigger.

  8. 34 minutes ago, mechanicalEYE said:

    It flashed the overheat warning on me around 22 minutes just sitting on the tripod with the display setting changed to 30minutes instead of 1 minute. No previous operation of the camera.

    It has overheated recording externally at 42 minutes. indoors and outdoors.

    Strangely I recorded 4K HQ externally yesterday in 102 degrees for 45 minutes straight with the camera blazing hot to the touch, ( hotter than any camera Ive touched ) brought the camera indoors while still recording because I was somewhat concerned about how hot it was, no overheat warning, all this was about 56 minutes and I'm sure it would have went past the hour mark easily. Camera was blazing hot and while it was recording I inserted the CFexpress card, opened the door, and the camera shutdown, once I inserted the card and closed the door the R5 went right back to recording to the ninja. I shut the ninja down and my R5 counter said 20 minutes of HQ, and a few moments later was at 25 minutes. The camera was way too hot, and I honestly expected to see minimal time allowed to record HQ and all other modes to the card, 4K 120, all flavors of 8K, with no reduction in times. Camera was still pretty warm to the touch in the end of it all. It was like I never turned it on. No freezer needed.

     

    Very interesting, thanks for the report. Your observations point to DIGIC's thermals being what the firmware is triggering for the thermal shutdown (ie, the fact the camera allowed itself to get very warm for external recording but then immediately shut down when the camera was configured to allow internal recording, ie CFE card inserted). Also, the fact the camera allows near-unlimited recording over HDMI argues against intentional crippling since external recorders are common for professional video use - if Canon wanted to steer pro video customers away from the R5 to their rumored upcoming RF-based cinema cameras I don't see how they would permit long-run recording over HDMI.

  9. 28 minutes ago, Andrew Reid said:

    It doesn't need it. You should just be able to turn it off and the extremely thin slice of silicon & metal will cool down by itself from whatever it couldn't handle... Supposably 90C... To ambient temp or a comfortable for any CPU 45C... In less than a minute, if not even quicker, we're talking a few seconds to go from 90 to 60.

    Anyone have a PC?

    Open the CPU temp monitor and see how fast the temps fall once you stop rendering video in an NLE.

    And they fall even faster if you pull the power completely.

    Computers are larger and aren't weather sealed. My laptop stays warm for 10 minutes after I shut it off after it's been running at load.

  10. 33 minutes ago, Andrew Reid said:

    Do us a favour and realise that the camera overheats in the menus or just doing stills in live-view.

    Overheats as far as a complete shut down once you go over into video mode.

    But yes... Video compression blah blah blah, data intensive blah blah.

    I have not seen any reports of the camera overheating to the point of shutting down from navigating within the menus or shooting stills. I have seen reports of the cumulative heat from those activities exhausting the video-recording heat budget. The camera can continue to provide video over HDMI during that period so I'm not seeing anything in your comment which refutes how video compression through DIGIC isn't a factor for the overheating that leads to a thermal shutdown.

  11. 2 hours ago, Video Hummus said:

     

    We can safety rule out the sensor heat as being the main culprit if people are able to record 4KHQ externally for hours on end.

    It is either something going on with the CFExpress cards getting to hot or some kind of timer that is started when they are inserted.

    The fact that the camera power cycles when the memory or battery doors open may be a hint that that may be the case.

    Please keep testing @Andrew Reid and @mechanicalEYE very helpful for people. 

    It's more likely to be DIGIC than CFE. External recording bypasses the entire video compression path, which is very computation and data-intensive (ie, generates a lot of heat). Plus the camera can recorder indefinitely to CFE in 4KLQ.

  12. 3 hours ago, ajay said:

    Get a case of it.

    In all seriousness, everyone so far has tried to cool the camera down externally and not internally. If you want to be absolutely, positively be sure that Canon is using the cripple hammer, it would be best to chill down the camera ASAP and retest record limits. Don't wait hours. Don't try putting it in the freezer. By getting the circuit boards chilled within minutes will give you a definitive answer to whether Canon is intentionally crippling the camera.

    Do we know for sure there is only one temperature sensor in the camera? Maybe Canon is reading the temperature of a different component other than the one in your experiment.

    I'm not trying to be a jerk about this, but we need a secondary test to prove beyond a doubt that Canon is intentionally crippling this camera. This is truly a big deal.

    Chilling the circuit boards directly and rapidly will prove this out.

    (Last post on this. I don't mean to keep repeating this but I do feel it's important. I'm done.)

    The crippling has already been disproven (at least the recovery-period aspect of it) - the poster on FM (link) was able to get the full available 8K video recording time after putting the camera in the freezer for just 25 minutes following a thermal shutdown.

  13. 2 hours ago, Andrew Reid said:

    Even if it turns out that the EXIF "camera temperature" has no relation to the temperature of the DIGIC X chip, the test results still confirm fake overheat timers. Why - let me explain:

    1. CDA TEK's (BTM_PIX)'s app reads the temperature inhibitors. If the EXIF temp sensor is on a separate board and is not reading the CPU core temps, it would still increase if the CPU got hotter. The CPU is the primary heat source in the camera body and heats up the entire unit. We show that the temp inhibitors toggle on over time, even as the EXIF reported temp stays level at 46C or even decreases (as well as camera body external temps to the touch).

    2. After powering off an electronic device or CPU, the part dissipates most of the 90C+ heat within seconds. If opening the card door, the chipset even then has a venting hole. Hot air escapes and dissipates quickly. When I switched off the camera for 1 hour and opened the card door, removed battery and lens, the temp inhibitors were left on even after 1 hour. It is impossible for the CPU to maintain 60C+ temps (33C above ambient temperature!) in this situation with no power.

    3. The temp inhibitors flag ON even when the camera is not recording. They flag ON in the Wifi menu. ON in live-view. ON during stills shooting. If these processes are causing problematic temperatures for the CPU, it should shut the camera down in stills mode, but it doesn't... it can be left on.

    Sure, but see my point 1 & 2 above.

    A separate temp sensor would not stay at 46C if a main CPU at 90C started heating the body and internal air.

    And heat distribution via air does not take 1 hour between a hot silicon chip and the ambient air right next to it and in/around the card slot.

    The Chinese heat gun shows this not to be the case, but anyway, if the sensor is the limiting part that gets too hot, it would have to shut down during an 8K (45 megapixel) photo shoot at the same time it shuts down for 8K video.

    1. There will be a point where both DIDIG/SDRAM and internal temps reach apparent homeostasis. Under constant processing load the hottest chips will heat up immediately and reach a plateau quickly, which will be based upon their thermal profile relative to ambient. The ambient temperature around the chips and inside the camera chamber will rise more slowly (hysteresis). As ambient continues to slowly rise the heat-generating chips will get hotter as well but on a slower ramp. This thermal feedback loop will slow over time as the camera's dissipation reaches homeostasis with the heat generation.

    2. After powering off the hottest chips will cool immediately, however the ambient temperature around the chips will not. When power and processing load is reapplied those chips will quickly heat back to their pre-shutdown temps since their temp is a function of heat generation relative to ambient temp.

    3. What you call "temp inhibitors" is a combination of the latent ambient temperature and logic that is projecting the ongoing heat ramp, which factors in the thermal lag affects described above (hysteresis).

    The fact the freezer experiment posted on Fred Miranda returned the camera to full video operation times in only 25 minutes serves to both disprove the "timer-based cripple" theory while also demonstrating how the thermal management algorithm is actually working. The logic is serving two purposes - 1) Keep all chips on the PCB within their thermal design limits and 2) Project how much time remains before the chips will reach their thermal limits. The logic for #2 must factor in thermal hysteresis, which to the untrained will may look like intentional crippling and temperature-indifference since the algorithm wont respond as immediately or definitively as one would expect based on instantaneous temps but in reality is actually projecting out a longer-duration thermal ramp. You can see the result of this calculation in the available video-time value reported to the user in the UI.

     

  14. 12 minutes ago, mechanicalEYE said:

    Yeah, I pulled battery while camera was recording. As soon as you open the battery door, camera shots off anyway. Pulled battery and then reinserted battery, available time did not change.

    Thanks. The fact the camera shuts off as soon as you open the battery door means it's using battery-door sensor to perform an orderly shutdown before you get a chance to remove the battery. Unfortunately that makes the experiment more difficult to execute. The only way around it would be to find where the battery door sensor is and manually hold the sensor in while the door is open, then do the video recording, then pull the battery.

    14 minutes ago, ntblowz said:

    and took battery out for too, wonder is that the reason to reset the timer?

    I think others have tried battery removals without any effect on subsequent available recording times. The difference in this test vs any others I've read is he kept the body in the freezer for 25 minutes. That's much more definitive in disproving the firmware cripple timer theory than putting ice on the camera or adding better heat sinks.

  15. 3 hours ago, mechanicalEYE said:

    I tried this.

    I turned the camera on let it sit at idle with no card in camera, once I got the overheat warning, I shut the camera down. 

    I then grabbed my 1TB cfexpress card put it in camera, and the time allowance for HQ recording read 10 minutes.

    Started a recording, changed aperture, and iso, as soon as you open the battery door camera shuts down, pulled battery and reinserted the battery.

    All values were as left at shut down, and 10 minute limit remained.

    Thanks. It's not clear you tried the specific experiment I outlined. The battery should be pulled while the camera is recording internally to the card (abrupt shutdown), somewhere near where it would thermally shutdown. Then the battery should be put back in and check what the available recording time the camera reports.

  16. On previous Canon bodies the temperature report in the EXIF is from what's called the "EFIC temp" (source), which is a secondary, low-speed chip responsible for the lens interface and speedlites. Canon actual documented this chip in their 1DM3 technical whitepaper (page 33), where Canon refers to it as follows "EFIC as the interface for the lens and external Speedlite" and explicitly states it's located on the camera control board, separate from the image sensor and DIGIC. If the R5 follows this arrangement this means the temperature in the EXIF is not of the sensor or DIGIC, which explains why it plateaus quickly and doesn't correlate to the video-recording thermal shutdown events, which instead are likely related to the temperature of DIGIC and/or the surrounding support chips like DDR.

  17. 1 hour ago, Andrew Reid said:

    Tried the clock and reset, hasn't worked so far but we'll keep probing.

     

    Try a battery pull while the camera is still on, both just after stopping a record and if that doesn't extend the subsequent recording time, while the video is actually recording. Check if the camera "remembers" the remaining time limit available when you plug the battery back in (details here).

  18. The best way to determine if the time limit is firmware-related is to attempt to defeat the firmware's mechanism to record the time/states associated with the thermal shutdown. All camera settings and firmware state variables are stored in NVRAM. The settings are typically only written out when the camera does an orderly shutdown (optimization). You can verify this yourself by changing one of the shooting parameters (aperture, shutter speed, ISO) and then pulling out the battery while the camera is on. When you plug the battery back in you'll likely see the camera didn't save that change. The state variables associated with any potential video timer/limit/thermal threshold might be defeated using this same technique.

  19. "Why does the Canon API documentation claim you are able to poll the temperature, but the camera only returns a text status like “Normal” rather than a real temperature in Celsius? Why is this all-important temperature readout obscured?"

    The precise internal temperature is available in the EXIF metadata of R5 images. If you'd like to experiment on thermal shutdown temperature thresholds/cooldown periods you can take a photo before/during/after the test and use the EXIF of those photos to monitor the temperatures over the period of the test. For example, here's an exiftool invocation to get the internal camera temperature for all R5 .CR3 raw files in a directory:

    exiftool.exe -CameraTemperature "c:\mypics\*.cr3"

     

     

     

  20. 6 hours ago, Video Hummus said:

    ...And I think this why Sony is slowly overtaking them.

    I honestly think Canon is just riding the consumer camera train with their brand and logo as the ticket for as long as possible with almost zero investment because there is not much profits in it anymore.

    When that train loses steam they will fall back on their camera patents and lens designs and license those.

    I find it hard to believe there is zero foresight at the management of this company. And don’t tell me it’s a Japanese thing. Other Japanese camera companies seem to get it. 

    Agree, the camera market has been collapsing for some time and I think Canon just chose to limit its R&D investment to maximize its profit and keep the cash-cow going for as long as they could. They might have been able to squeeze some additional unit sales by targeting hybrid/video shooters with less-crippled offerings but they probably decided it wasn't useful since the market is seen falling into a hyper-competitive, lower-margin business anyway, with insufficient volume on the low end to amortize their R&D and manufacturing capital investments. Such is not the case for their professional video division so crippling the video on the prosumer/enthusiast offerings was probably seen as an additional safeguard against cannibalization of sales of that division.

  21. On 5/20/2018 at 7:34 AM, KnightsFan said:

    Exactly. So my intuition was that a 17.5mm f0.95 on MFT would have roughly the same angle of incidence as a 35mm f2.0 on FF, thus making pixel vignetting a non-issue when comparing the two.

    I don't see what you're basing that intuition/assumption on. There is no equivalence theory to apply here - optics/ray angles don't change based on sensor size. The only impact on sensor size would be using a larger-format lens on a smaller sensor, where some of the rays are thus cropped (don't reach the image sensor), but that's not what we're speaking to here - the Voigtlander is a native lens.

  22. On 5/18/2018 at 6:03 PM, KnightsFan said:

    Same reason why there is less vignetting on a center crop from a full frame image. Angle of incidence becomes more oblique the farther from the center of the image.

    But we're talking about a native MFT lens (Voigtlander 17.5mm f/0.95), so the relative angle of incidence across the frame will be the same as a FF sensor with a native FF lens of a similarly scaled design.

×
×
  • Create New...