Jump to content

horshack

Members
  • Posts

    167
  • Joined

  • Last visited

Reputation Activity

  1. Like
    horshack got a reaction from Emanuel in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    The goal isn't to save the parameters...the goal is to *not* save them 😀 It's the entire basis of the workaround for the timer element of the R5's thermal algorithm.
    Btw, you don't have to turn off the camera to save the parameters. Taking a photo or a video or moving the mode dial between stills/movie mode saves them as well.
  2. Like
    horshack got a reaction from ajay in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    Update:
    I recorded 25:00 of video then battery pull with no issue on my RP. All ~4GB MP4 segments were fine, with naturally the last one missing Don't let the recording reach the 29:59 legacy limit - if you do the camera will save the NVRAM settings when it automatically ends the recording The Ridgecrop formatter tool link I posted for Windows only worked once for me. After that the camera wouldn't accept any cards formatted with it. I then tried the formatter built into OSX using the FAT32 and MBR options (not GPT) - the camera accepts those cards but then displays an error whenever you take a photo or end a video recording. The only reliable and consistent way I've found to format > 32GB cards for FAT32 that my RP would accept and work with is under Linux. You can see instructions how to do so here. Here's a screenshot of the ~4GB segements on a 64GB SD card formatted to FAT32 for my 25:00 recording:
  3. Like
    horshack got a reaction from docmoore in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    Update:
    I recorded 25:00 of video then battery pull with no issue on my RP. All ~4GB MP4 segments were fine, with naturally the last one missing Don't let the recording reach the 29:59 legacy limit - if you do the camera will save the NVRAM settings when it automatically ends the recording The Ridgecrop formatter tool link I posted for Windows only worked once for me. After that the camera wouldn't accept any cards formatted with it. I then tried the formatter built into OSX using the FAT32 and MBR options (not GPT) - the camera accepts those cards but then displays an error whenever you take a photo or end a video recording. The only reliable and consistent way I've found to format > 32GB cards for FAT32 that my RP would accept and work with is under Linux. You can see instructions how to do so here. Here's a screenshot of the ~4GB segements on a 64GB SD card formatted to FAT32 for my 25:00 recording:
  4. Like
    horshack got a reaction from ac6000cw in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    Update:
    I recorded 25:00 of video then battery pull with no issue on my RP. All ~4GB MP4 segments were fine, with naturally the last one missing Don't let the recording reach the 29:59 legacy limit - if you do the camera will save the NVRAM settings when it automatically ends the recording The Ridgecrop formatter tool link I posted for Windows only worked once for me. After that the camera wouldn't accept any cards formatted with it. I then tried the formatter built into OSX using the FAT32 and MBR options (not GPT) - the camera accepts those cards but then displays an error whenever you take a photo or end a video recording. The only reliable and consistent way I've found to format > 32GB cards for FAT32 that my RP would accept and work with is under Linux. You can see instructions how to do so here. Here's a screenshot of the ~4GB segements on a 64GB SD card formatted to FAT32 for my 25:00 recording:
  5. Like
    horshack got a reaction from kaylee in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    You sir win the internet today. I just tried it on my Canon RP, which like the R5 forgets any changes to the camera's settings (like exposure settings) if you pull the battery in the middle of the recording but will remember the changed settings if you stop a recording normally.
    Procedure:
    Tape down the door sensor and leave the door open so the battery is accessible Put a 32GB card in the camera and Turn the camera on and format the card. It will automatically use FAT32 cards with a capacity <= 32GB Start a video recording, in this case I used 4K 24P, which has an average total video/audio bitrate of 14.43 MB/s. Shoot a video whose length will exceed 4GB. For my 4K 24P that's any video longer than just under 5:00. I shot one for exactly 5:00 Pull the battery Check the card and verify there's a single .MP4 file Reinsert the battery and verify the camera "forgot" the previous session's settings We know the camera saves its settings whenever it has successfully recorded a video. The question is whether @Jn- idea will avoid that save for the FAT32-case of splitting up a video file at 4GB/each (FAT32 limit), provided we still pull the battery at the end of the recording. In other words, will the camera save NVRAM for each intermediate 4GB-split file or only for the final file after properly stopping a recording?
    Drumroll...It works! My RP's aperture was f/4 when I turned it on for step #3. I changed it to f/11 for the test. After reinserting the battery my RP's aperture reverted back to f/4!
    Andrew, please test this ASAP on your R5 and see if it matches my RP's behavior of forgetting the settings, the most important of which is of course the thermal timer.
     
  6. Thanks
    horshack got a reaction from Emanuel in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    You sir win the internet today. I just tried it on my Canon RP, which like the R5 forgets any changes to the camera's settings (like exposure settings) if you pull the battery in the middle of the recording but will remember the changed settings if you stop a recording normally.
    Procedure:
    Tape down the door sensor and leave the door open so the battery is accessible Put a 32GB card in the camera and Turn the camera on and format the card. It will automatically use FAT32 cards with a capacity <= 32GB Start a video recording, in this case I used 4K 24P, which has an average total video/audio bitrate of 14.43 MB/s. Shoot a video whose length will exceed 4GB. For my 4K 24P that's any video longer than just under 5:00. I shot one for exactly 5:00 Pull the battery Check the card and verify there's a single .MP4 file Reinsert the battery and verify the camera "forgot" the previous session's settings We know the camera saves its settings whenever it has successfully recorded a video. The question is whether @Jn- idea will avoid that save for the FAT32-case of splitting up a video file at 4GB/each (FAT32 limit), provided we still pull the battery at the end of the recording. In other words, will the camera save NVRAM for each intermediate 4GB-split file or only for the final file after properly stopping a recording?
    Drumroll...It works! My RP's aperture was f/4 when I turned it on for step #3. I changed it to f/11 for the test. After reinserting the battery my RP's aperture reverted back to f/4!
    Andrew, please test this ASAP on your R5 and see if it matches my RP's behavior of forgetting the settings, the most important of which is of course the thermal timer.
     
  7. Thanks
    horshack got a reaction from Emanuel in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    Just tried it for a longer video, this time 10:00, to test two splits and it still works. Two perfectly-good .MP4 files, each approx 4GB and 4:41 in length, and an RP with amnesia after powering it back on.
    The only limitation of this technique is 32GB, since Canon cameras format all cards > 32GB as exFAT, even though FAT32 supports volumes up to 2TB. But there's a workaround for that as well - format the >32GB card on computer instead. Windows 7/10 wont let you format cards > 32GB to FAT32 but there are third-party utilities that will. I just used http://www.ridgecrop.demon.co.uk/index.htm?guiformat.htm on a 64GB card and the camera accepted the card with no complaints. Recording to it right now on my RP to test splits for a very long video. And while Windows wont let you format >32GB cards to FAT32 it will mount them just fine.
  8. Like
    horshack got a reaction from Stathman in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    Just tried it for a longer video, this time 10:00, to test two splits and it still works. Two perfectly-good .MP4 files, each approx 4GB and 4:41 in length, and an RP with amnesia after powering it back on.
    The only limitation of this technique is 32GB, since Canon cameras format all cards > 32GB as exFAT, even though FAT32 supports volumes up to 2TB. But there's a workaround for that as well - format the >32GB card on computer instead. Windows 7/10 wont let you format cards > 32GB to FAT32 but there are third-party utilities that will. I just used http://www.ridgecrop.demon.co.uk/index.htm?guiformat.htm on a 64GB card and the camera accepted the card with no complaints. Recording to it right now on my RP to test splits for a very long video. And while Windows wont let you format >32GB cards to FAT32 it will mount them just fine.
  9. Like
    horshack got a reaction from Stathman in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    You sir win the internet today. I just tried it on my Canon RP, which like the R5 forgets any changes to the camera's settings (like exposure settings) if you pull the battery in the middle of the recording but will remember the changed settings if you stop a recording normally.
    Procedure:
    Tape down the door sensor and leave the door open so the battery is accessible Put a 32GB card in the camera and Turn the camera on and format the card. It will automatically use FAT32 cards with a capacity <= 32GB Start a video recording, in this case I used 4K 24P, which has an average total video/audio bitrate of 14.43 MB/s. Shoot a video whose length will exceed 4GB. For my 4K 24P that's any video longer than just under 5:00. I shot one for exactly 5:00 Pull the battery Check the card and verify there's a single .MP4 file Reinsert the battery and verify the camera "forgot" the previous session's settings We know the camera saves its settings whenever it has successfully recorded a video. The question is whether @Jn- idea will avoid that save for the FAT32-case of splitting up a video file at 4GB/each (FAT32 limit), provided we still pull the battery at the end of the recording. In other words, will the camera save NVRAM for each intermediate 4GB-split file or only for the final file after properly stopping a recording?
    Drumroll...It works! My RP's aperture was f/4 when I turned it on for step #3. I changed it to f/11 for the test. After reinserting the battery my RP's aperture reverted back to f/4!
    Andrew, please test this ASAP on your R5 and see if it matches my RP's behavior of forgetting the settings, the most important of which is of course the thermal timer.
     
  10. Like
    horshack got a reaction from docmoore in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    Just tried it for a longer video, this time 10:00, to test two splits and it still works. Two perfectly-good .MP4 files, each approx 4GB and 4:41 in length, and an RP with amnesia after powering it back on.
    The only limitation of this technique is 32GB, since Canon cameras format all cards > 32GB as exFAT, even though FAT32 supports volumes up to 2TB. But there's a workaround for that as well - format the >32GB card on computer instead. Windows 7/10 wont let you format cards > 32GB to FAT32 but there are third-party utilities that will. I just used http://www.ridgecrop.demon.co.uk/index.htm?guiformat.htm on a 64GB card and the camera accepted the card with no complaints. Recording to it right now on my RP to test splits for a very long video. And while Windows wont let you format >32GB cards to FAT32 it will mount them just fine.
  11. Like
    horshack got a reaction from docmoore in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    I think I've found out why the video playback is blank and the there's no audio. The MP4 header has 'stsz' and 'stco' atoms that are indexes into the actual video and audio data in the mdat atom. Those atoms are used by players/NLEs to correlate time to video/audio frames. Since I copied the entire MP4 header from the "good" video those indexes will be wrong for the "bad" video and have to be rebuilt. I found an open-source tool named uncut that does this (https://github.com/ponchio/untrunc), documentation here. Built and ran it but it wasn't able to find any of the frames in the files. Will be looking into the code to see if it can be tweaked to work on these files
  12. Like
    horshack got a reaction from hyalinejim in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    Just tried it for a longer video, this time 10:00, to test two splits and it still works. Two perfectly-good .MP4 files, each approx 4GB and 4:41 in length, and an RP with amnesia after powering it back on.
    The only limitation of this technique is 32GB, since Canon cameras format all cards > 32GB as exFAT, even though FAT32 supports volumes up to 2TB. But there's a workaround for that as well - format the >32GB card on computer instead. Windows 7/10 wont let you format cards > 32GB to FAT32 but there are third-party utilities that will. I just used http://www.ridgecrop.demon.co.uk/index.htm?guiformat.htm on a 64GB card and the camera accepted the card with no complaints. Recording to it right now on my RP to test splits for a very long video. And while Windows wont let you format >32GB cards to FAT32 it will mount them just fine.
  13. Like
    horshack got a reaction from mechanicalEYE in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    I think I've found out why the video playback is blank and the there's no audio. The MP4 header has 'stsz' and 'stco' atoms that are indexes into the actual video and audio data in the mdat atom. Those atoms are used by players/NLEs to correlate time to video/audio frames. Since I copied the entire MP4 header from the "good" video those indexes will be wrong for the "bad" video and have to be rebuilt. I found an open-source tool named uncut that does this (https://github.com/ponchio/untrunc), documentation here. Built and ran it but it wasn't able to find any of the frames in the files. Will be looking into the code to see if it can be tweaked to work on these files
  14. Like
    horshack got a reaction from Andrew Reid in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    I think I've found out why the video playback is blank and the there's no audio. The MP4 header has 'stsz' and 'stco' atoms that are indexes into the actual video and audio data in the mdat atom. Those atoms are used by players/NLEs to correlate time to video/audio frames. Since I copied the entire MP4 header from the "good" video those indexes will be wrong for the "bad" video and have to be rebuilt. I found an open-source tool named uncut that does this (https://github.com/ponchio/untrunc), documentation here. Built and ran it but it wasn't able to find any of the frames in the files. Will be looking into the code to see if it can be tweaked to work on these files
  15. Like
    horshack got a reaction from PaulUsher in My Canon EOS R5 recording 8K video 50 minutes straight   
    Petapixel has picked it up:
    https://petapixel.com/2020/08/24/simple-hack-proves-canon-eos-r5-overheating-limit-is-artificial/
  16. Thanks
    horshack got a reaction from docmoore in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    Made a lot of progress on understanding the .DAT files. Using my Canon RP as a test bed, I recorded two 1:00 videos, one regularly and the other power-interrupted. I then copied the entire SD card for each case as a file image to my system for analysis. Prior to recording video each SD was prepared by fully wiping it with dd, then formatting in the camera. I also went back to FAT32 because it's easier to analyze those structures than exFAT.
    This yields two very similar filesystem images. For the regularly-recorded video SD image I see a single data file (MVI_0478.MP4 in this specific case). However if I analyze the directory entries in the FAT32 structure I see a second file that's been deleted, named ?VI_047.DAT. The '?' is there because the file has been deleted (0xE5 is the FAT32 marker to indicate a deleted directory entry), meaning its original name is MVI_0478.DAT. Even though MVI_0478.DAT has been deleted its FAT32 cluster location and most-recent file size is still available in the directory entry.
    Looking at the contents of the .DAT I see an "mdat" field. This is the raw MP4 data holding the video/data of the video - it's called an "mdat atom" in MP4 container parlance. The size of the .DAT file is exactly 128KB less than the size of the .MP4. What's more, the on-SD location of the .DAT is exactly 128KB after the .MP4, which means the two files are actually contiguous with each other.
    So the camera wrote the .DAT first (while recording the video) but left 128KB before it pre-allocated in the FAT32 cluster table. When the video recording stopped it then wrote the 128KB before the .DAT, which contains all the other MP4 atoms necessary to represent the file, including the 'moov atom' which was reported missing when trying to recover the .DAT as .MP4 earlier using open-source recovery tools (failed because it's missing the full 128KB .MP4 header).
    The 128KB MP4 container header stuff + mdat atom content represents the full MP4 file.
    So the .DAT files we're seeing for the interrupted recordings on the R5 are the actual raw video/audio data sans the proper 128KB MP4 header area.
    On my Canon RP the .DAT directory entry is missing for the interrupt recording. However if I go to the location on the interrupted-SD image as where the .DAT file exists on the good-SD image I see the mdat data. So it appears that on my RP the camera writes to this area during recording without first setting the directory entry to reflect that area is allocated. If you're able to see the .DAT files from your 1DX that means the behavior of that camera is different. Not sure how the R5 acts. Either way, recovering the .DAT file for my RP is trivial because I know what cluster it begins on by comparing it to the good-SD image.
    I found the .DAT area on the interrupted-SD image and spliced in the 128KB MP4 header from the .MP4 from the good image. The file plays...and I hear a quick snippet of sound, but no video. So there is work to do there. But at least it demonstrates the basic integrity of the full MP4 is in place.
    To get this working on R5 images we'd have to determine how to build the proper 128KB MP4 container header, the same as how the camera builds it when the user ends a video recording. This might be as simple as splicing from a "good" .MP4 image, but probably requires a lot more effort, and then graft that 128KB header data with the the .DAT file to reconstruct the full .MP4
  17. Thanks
    horshack got a reaction from Stathman in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    Made a lot of progress on understanding the .DAT files. Using my Canon RP as a test bed, I recorded two 1:00 videos, one regularly and the other power-interrupted. I then copied the entire SD card for each case as a file image to my system for analysis. Prior to recording video each SD was prepared by fully wiping it with dd, then formatting in the camera. I also went back to FAT32 because it's easier to analyze those structures than exFAT.
    This yields two very similar filesystem images. For the regularly-recorded video SD image I see a single data file (MVI_0478.MP4 in this specific case). However if I analyze the directory entries in the FAT32 structure I see a second file that's been deleted, named ?VI_047.DAT. The '?' is there because the file has been deleted (0xE5 is the FAT32 marker to indicate a deleted directory entry), meaning its original name is MVI_0478.DAT. Even though MVI_0478.DAT has been deleted its FAT32 cluster location and most-recent file size is still available in the directory entry.
    Looking at the contents of the .DAT I see an "mdat" field. This is the raw MP4 data holding the video/data of the video - it's called an "mdat atom" in MP4 container parlance. The size of the .DAT file is exactly 128KB less than the size of the .MP4. What's more, the on-SD location of the .DAT is exactly 128KB after the .MP4, which means the two files are actually contiguous with each other.
    So the camera wrote the .DAT first (while recording the video) but left 128KB before it pre-allocated in the FAT32 cluster table. When the video recording stopped it then wrote the 128KB before the .DAT, which contains all the other MP4 atoms necessary to represent the file, including the 'moov atom' which was reported missing when trying to recover the .DAT as .MP4 earlier using open-source recovery tools (failed because it's missing the full 128KB .MP4 header).
    The 128KB MP4 container header stuff + mdat atom content represents the full MP4 file.
    So the .DAT files we're seeing for the interrupted recordings on the R5 are the actual raw video/audio data sans the proper 128KB MP4 header area.
    On my Canon RP the .DAT directory entry is missing for the interrupt recording. However if I go to the location on the interrupted-SD image as where the .DAT file exists on the good-SD image I see the mdat data. So it appears that on my RP the camera writes to this area during recording without first setting the directory entry to reflect that area is allocated. If you're able to see the .DAT files from your 1DX that means the behavior of that camera is different. Not sure how the R5 acts. Either way, recovering the .DAT file for my RP is trivial because I know what cluster it begins on by comparing it to the good-SD image.
    I found the .DAT area on the interrupted-SD image and spliced in the 128KB MP4 header from the .MP4 from the good image. The file plays...and I hear a quick snippet of sound, but no video. So there is work to do there. But at least it demonstrates the basic integrity of the full MP4 is in place.
    To get this working on R5 images we'd have to determine how to build the proper 128KB MP4 container header, the same as how the camera builds it when the user ends a video recording. This might be as simple as splicing from a "good" .MP4 image, but probably requires a lot more effort, and then graft that 128KB header data with the the .DAT file to reconstruct the full .MP4
  18. Thanks
    horshack got a reaction from visionrouge in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    Made a lot of progress on understanding the .DAT files. Using my Canon RP as a test bed, I recorded two 1:00 videos, one regularly and the other power-interrupted. I then copied the entire SD card for each case as a file image to my system for analysis. Prior to recording video each SD was prepared by fully wiping it with dd, then formatting in the camera. I also went back to FAT32 because it's easier to analyze those structures than exFAT.
    This yields two very similar filesystem images. For the regularly-recorded video SD image I see a single data file (MVI_0478.MP4 in this specific case). However if I analyze the directory entries in the FAT32 structure I see a second file that's been deleted, named ?VI_047.DAT. The '?' is there because the file has been deleted (0xE5 is the FAT32 marker to indicate a deleted directory entry), meaning its original name is MVI_0478.DAT. Even though MVI_0478.DAT has been deleted its FAT32 cluster location and most-recent file size is still available in the directory entry.
    Looking at the contents of the .DAT I see an "mdat" field. This is the raw MP4 data holding the video/data of the video - it's called an "mdat atom" in MP4 container parlance. The size of the .DAT file is exactly 128KB less than the size of the .MP4. What's more, the on-SD location of the .DAT is exactly 128KB after the .MP4, which means the two files are actually contiguous with each other.
    So the camera wrote the .DAT first (while recording the video) but left 128KB before it pre-allocated in the FAT32 cluster table. When the video recording stopped it then wrote the 128KB before the .DAT, which contains all the other MP4 atoms necessary to represent the file, including the 'moov atom' which was reported missing when trying to recover the .DAT as .MP4 earlier using open-source recovery tools (failed because it's missing the full 128KB .MP4 header).
    The 128KB MP4 container header stuff + mdat atom content represents the full MP4 file.
    So the .DAT files we're seeing for the interrupted recordings on the R5 are the actual raw video/audio data sans the proper 128KB MP4 header area.
    On my Canon RP the .DAT directory entry is missing for the interrupt recording. However if I go to the location on the interrupted-SD image as where the .DAT file exists on the good-SD image I see the mdat data. So it appears that on my RP the camera writes to this area during recording without first setting the directory entry to reflect that area is allocated. If you're able to see the .DAT files from your 1DX that means the behavior of that camera is different. Not sure how the R5 acts. Either way, recovering the .DAT file for my RP is trivial because I know what cluster it begins on by comparing it to the good-SD image.
    I found the .DAT area on the interrupted-SD image and spliced in the 128KB MP4 header from the .MP4 from the good image. The file plays...and I hear a quick snippet of sound, but no video. So there is work to do there. But at least it demonstrates the basic integrity of the full MP4 is in place.
    To get this working on R5 images we'd have to determine how to build the proper 128KB MP4 container header, the same as how the camera builds it when the user ends a video recording. This might be as simple as splicing from a "good" .MP4 image, but probably requires a lot more effort, and then graft that 128KB header data with the the .DAT file to reconstruct the full .MP4
  19. Like
    horshack got a reaction from Andrew Reid in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    Thanks for the files. Tried two separate automated MP4 repair programs/scripts, both of which support reading the headers/metdata from the "good" file and attempt to graft that data over those missing elements in the bad file, specifically the missing moov atom in the corrupt file. No joy so far. Oddly when I examine the "good" file the moov atom is actually at the beginning of the file - I expected it to be at the end since that's what's missing from the corrupt file. It might be that Canon relocates the moov atom to the front after a successful recording end. It's usually placed at the beginning of files to facilitate better streaming as the file is being downloaded. Or it might be that IsoBuster didn't recover the file correctly. I'll keep digging.
    Here's the Atom data from the good file, obtained from AtomicParsley <filename> -T
    Atom ftyp @ 0 of size: 28, ends @ 28
    Atom moov @ 28 of size: 262116, ends @ 262144
         Atom uuid=85c0b687-820f-11e0-8111-f4ce462b6a48 @ 36 of size: 65750, ends @ 65786
         Atom udta @ 65786 of size: 2056, ends @ 67842
             Atom manu @ 65794 of size: 20, ends @ 65814                     ~
             Atom modl @ 65814 of size: 38, ends @ 65852                     ~
             Atom urat @ 65852 of size: 16, ends @ 65868                     ~
             Atom free @ 65868 of size: 1974, ends @ 67842
         Atom mvhd @ 67842 of size: 108, ends @ 67950
         Atom trak @ 67950 of size: 16328, ends @ 84278
             Atom tkhd @ 67958 of size: 92, ends @ 68050
             Atom edts @ 68050 of size: 36, ends @ 68086
                 Atom elst @ 68058 of size: 28, ends @ 68086
             Atom mdia @ 68086 of size: 16192, ends @ 84278
                 Atom mdhd @ 68094 of size: 32, ends @ 68126
                 Atom hdlr @ 68126 of size: 33, ends @ 68159
                 Atom minf @ 68159 of size: 16119, ends @ 84278
                     Atom vmhd @ 68167 of size: 20, ends @ 68187
                     Atom dinf @ 68187 of size: 36, ends @ 68223
                         Atom dref @ 68195 of size: 28, ends @ 68223
                     Atom stbl @ 68223 of size: 16055, ends @ 84278
                         Atom stsd @ 68231 of size: 311, ends @ 68542
                             Atom hvc1 @ 68247 of size: 295, ends @ 68542             ~
                         Atom stts @ 68542 of size: 24, ends @ 68566
                         Atom stss @ 68566 of size: 296, ends @ 68862
                         Atom ctts @ 68862 of size: 4472, ends @ 73334
                         Atom stsc @ 73334 of size: 28, ends @ 73362
                         Atom stsz @ 73362 of size: 3364, ends @ 76726
                         Atom co64 @ 76726 of size: 6704, ends @ 83430
                         Atom sdtp @ 83430 of size: 848, ends @ 84278
         Atom trak @ 84278 of size: 8276, ends @ 92554
             Atom tkhd @ 84286 of size: 92, ends @ 84378
             Atom edts @ 84378 of size: 36, ends @ 84414
                 Atom elst @ 84386 of size: 28, ends @ 84414
             Atom mdia @ 84414 of size: 8140, ends @ 92554
                 Atom mdhd @ 84422 of size: 32, ends @ 84454
                 Atom hdlr @ 84454 of size: 33, ends @ 84487
                 Atom minf @ 84487 of size: 8067, ends @ 92554
                     Atom smhd @ 84495 of size: 16, ends @ 84511
                     Atom dinf @ 84511 of size: 36, ends @ 84547
                         Atom dref @ 84519 of size: 28, ends @ 84547
                     Atom stbl @ 84547 of size: 8007, ends @ 92554
                         Atom stsd @ 84555 of size: 103, ends @ 84658
                             Atom mp4a @ 84571 of size: 87, ends @ 84658
                                 Atom esds @ 84607 of size: 51, ends @ 84658
                         Atom stts @ 84658 of size: 24, ends @ 84682
                         Atom stsc @ 84682 of size: 748, ends @ 85430
                         Atom stsz @ 85430 of size: 6548, ends @ 91978
                         Atom co64 @ 91978 of size: 576, ends @ 92554
         Atom trak @ 92554 of size: 947, ends @ 93501
             Atom tkhd @ 92562 of size: 92, ends @ 92654
             Atom mdia @ 92654 of size: 847, ends @ 93501
                 Atom mdhd @ 92662 of size: 32, ends @ 92694
                 Atom hdlr @ 92694 of size: 33, ends @ 92727
                 Atom minf @ 92727 of size: 774, ends @ 93501
                     Atom nmhd @ 92735 of size: 12, ends @ 92747
                     Atom dinf @ 92747 of size: 36, ends @ 92783
                         Atom dref @ 92755 of size: 28, ends @ 92783
                     Atom stbl @ 92783 of size: 718, ends @ 93501
                         Atom stsd @ 92791 of size: 50, ends @ 92841
                             Atom tmcd @ 92807 of size: 34, ends @ 92841
                         Atom stts @ 92841 of size: 24, ends @ 92865
                         Atom stsc @ 92865 of size: 40, ends @ 92905
                         Atom stsz @ 92905 of size: 20, ends @ 92925
                         Atom co64 @ 92925 of size: 576, ends @ 93501
         Atom free @ 93501 of size: 168643, ends @ 262144
    Atom mdat @ 262144 of size: 730194240 (^), ends @ 0
                 (^)denotes a 64-bit atom length
     ~ denotes an unknown atom
    ------------------------------------------------------
    Total size: 730456384 bytes; 66 atoms total. AtomicParsley version: 0.9.0 (utf16)
    Media data: 730194240 bytes; 262144 bytes all other atoms (0.036% atom overhead).
    Total free atom space: 170617 bytes; 0.023% waste. Padding available: 1974 bytes.
    ------------------------------------------------------
  20. Like
    horshack got a reaction from docmoore in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    Thanks for the files. Tried two separate automated MP4 repair programs/scripts, both of which support reading the headers/metdata from the "good" file and attempt to graft that data over those missing elements in the bad file, specifically the missing moov atom in the corrupt file. No joy so far. Oddly when I examine the "good" file the moov atom is actually at the beginning of the file - I expected it to be at the end since that's what's missing from the corrupt file. It might be that Canon relocates the moov atom to the front after a successful recording end. It's usually placed at the beginning of files to facilitate better streaming as the file is being downloaded. Or it might be that IsoBuster didn't recover the file correctly. I'll keep digging.
    Here's the Atom data from the good file, obtained from AtomicParsley <filename> -T
    Atom ftyp @ 0 of size: 28, ends @ 28
    Atom moov @ 28 of size: 262116, ends @ 262144
         Atom uuid=85c0b687-820f-11e0-8111-f4ce462b6a48 @ 36 of size: 65750, ends @ 65786
         Atom udta @ 65786 of size: 2056, ends @ 67842
             Atom manu @ 65794 of size: 20, ends @ 65814                     ~
             Atom modl @ 65814 of size: 38, ends @ 65852                     ~
             Atom urat @ 65852 of size: 16, ends @ 65868                     ~
             Atom free @ 65868 of size: 1974, ends @ 67842
         Atom mvhd @ 67842 of size: 108, ends @ 67950
         Atom trak @ 67950 of size: 16328, ends @ 84278
             Atom tkhd @ 67958 of size: 92, ends @ 68050
             Atom edts @ 68050 of size: 36, ends @ 68086
                 Atom elst @ 68058 of size: 28, ends @ 68086
             Atom mdia @ 68086 of size: 16192, ends @ 84278
                 Atom mdhd @ 68094 of size: 32, ends @ 68126
                 Atom hdlr @ 68126 of size: 33, ends @ 68159
                 Atom minf @ 68159 of size: 16119, ends @ 84278
                     Atom vmhd @ 68167 of size: 20, ends @ 68187
                     Atom dinf @ 68187 of size: 36, ends @ 68223
                         Atom dref @ 68195 of size: 28, ends @ 68223
                     Atom stbl @ 68223 of size: 16055, ends @ 84278
                         Atom stsd @ 68231 of size: 311, ends @ 68542
                             Atom hvc1 @ 68247 of size: 295, ends @ 68542             ~
                         Atom stts @ 68542 of size: 24, ends @ 68566
                         Atom stss @ 68566 of size: 296, ends @ 68862
                         Atom ctts @ 68862 of size: 4472, ends @ 73334
                         Atom stsc @ 73334 of size: 28, ends @ 73362
                         Atom stsz @ 73362 of size: 3364, ends @ 76726
                         Atom co64 @ 76726 of size: 6704, ends @ 83430
                         Atom sdtp @ 83430 of size: 848, ends @ 84278
         Atom trak @ 84278 of size: 8276, ends @ 92554
             Atom tkhd @ 84286 of size: 92, ends @ 84378
             Atom edts @ 84378 of size: 36, ends @ 84414
                 Atom elst @ 84386 of size: 28, ends @ 84414
             Atom mdia @ 84414 of size: 8140, ends @ 92554
                 Atom mdhd @ 84422 of size: 32, ends @ 84454
                 Atom hdlr @ 84454 of size: 33, ends @ 84487
                 Atom minf @ 84487 of size: 8067, ends @ 92554
                     Atom smhd @ 84495 of size: 16, ends @ 84511
                     Atom dinf @ 84511 of size: 36, ends @ 84547
                         Atom dref @ 84519 of size: 28, ends @ 84547
                     Atom stbl @ 84547 of size: 8007, ends @ 92554
                         Atom stsd @ 84555 of size: 103, ends @ 84658
                             Atom mp4a @ 84571 of size: 87, ends @ 84658
                                 Atom esds @ 84607 of size: 51, ends @ 84658
                         Atom stts @ 84658 of size: 24, ends @ 84682
                         Atom stsc @ 84682 of size: 748, ends @ 85430
                         Atom stsz @ 85430 of size: 6548, ends @ 91978
                         Atom co64 @ 91978 of size: 576, ends @ 92554
         Atom trak @ 92554 of size: 947, ends @ 93501
             Atom tkhd @ 92562 of size: 92, ends @ 92654
             Atom mdia @ 92654 of size: 847, ends @ 93501
                 Atom mdhd @ 92662 of size: 32, ends @ 92694
                 Atom hdlr @ 92694 of size: 33, ends @ 92727
                 Atom minf @ 92727 of size: 774, ends @ 93501
                     Atom nmhd @ 92735 of size: 12, ends @ 92747
                     Atom dinf @ 92747 of size: 36, ends @ 92783
                         Atom dref @ 92755 of size: 28, ends @ 92783
                     Atom stbl @ 92783 of size: 718, ends @ 93501
                         Atom stsd @ 92791 of size: 50, ends @ 92841
                             Atom tmcd @ 92807 of size: 34, ends @ 92841
                         Atom stts @ 92841 of size: 24, ends @ 92865
                         Atom stsc @ 92865 of size: 40, ends @ 92905
                         Atom stsz @ 92905 of size: 20, ends @ 92925
                         Atom co64 @ 92925 of size: 576, ends @ 93501
         Atom free @ 93501 of size: 168643, ends @ 262144
    Atom mdat @ 262144 of size: 730194240 (^), ends @ 0
                 (^)denotes a 64-bit atom length
     ~ denotes an unknown atom
    ------------------------------------------------------
    Total size: 730456384 bytes; 66 atoms total. AtomicParsley version: 0.9.0 (utf16)
    Media data: 730194240 bytes; 262144 bytes all other atoms (0.036% atom overhead).
    Total free atom space: 170617 bytes; 0.023% waste. Padding available: 1974 bytes.
    ------------------------------------------------------
  21. Like
    horshack reacted to docmoore in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    Just ran a short clip on my 1DXIII pulled the battery and extracted the DAT file to my desktop. ISObuster attempted to convert it to mpg but the resulting file
    was unplayable in VLC, unrecognized by FF:Works and Davinci Resolve Studio. The conversion in ISObuster was at 2.1MB/s ... so even if it were readable the length of time
    to convert would be untenable.
  22. Like
    horshack got a reaction from docmoore in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    Good call @docmoore. Just downloaded IsoBuster and it found the .mp4. It wont let me attempt to extract it without buying a license.  I'll search out a few other free/open-source solutions first before biting the bullet on the purchase.
    Here's a screenshot of the recovery:

  23. Thanks
    horshack got a reaction from Jerome Chiu in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    I looked into what it might take to recover the lost/missing video file after a battery pull. I reproduced the R5 hack scenario on my Canon RP by recording a minute of video on a freshly-formatted SD card and then pulling the battery while the door-sensor was inhibited. Like what Andrew discovered on his R5 the file is not corrupt but actually completely missing, both when attempting to play the video on the camera and when mounting the card on a computer. I then tried running SanDisk's RescuePro Deluxe and after scanning the entire card it didn't find the video.
    Here's what I think is happening. Obviously Canon has to write the video data to the card while recording, since it has no other place to retain it beyond the internal SDRAM buffer capacity. You can see easy evidence of this by watching the card access light continuously flicker. The strategy Canon is likely employing is to defer the writing of the official FAT metadata until the recording is orderly stopped. This would include elements such as the FAT directory entry, which anchors the file, and possibly also the linked FAT allocation tables. This would explain why RescuePro couldn't find the file, since there isn't enough or any recognizable metadata to reconstruct the orphaned data sectors associated with the video. Canon is either caching that information in memory until the file is closed or is writing it somewhere on the card that isn't linked to the official FAT structures. The benefit of this strategy, assuming they're using it, is that it prevents any potential filesystem-level corruption from incomplete metadata updates, since none of the official metadata structures are updated to link to the file until it's closed. The downside of this strategy is that it orphans all the data from the file and makes recovery more complicated.
    There are a few strategies in devising a recovery app for this situation. The first would be to reverse-engineer exactly what if any metadata Canon is writing during the recording and to where. If that orphaned metadata is in the same format as actual FAT structures then it should be relatively straightforward to create a placeholder directory entry and link to it. If the format of that metadata is proprietary to Canon then the process of reverse-engineering its structure would be much more complex. This would all be done by using a sector editor and block-based search tools to compare a freshly-formatted card to one which has a missing video after recording. And lots of effort.
    I was actually an embedded firmware storage engineer for most of my professional career so this would be in my wheelhouse. I'll see how much demand there might be for this before I spent any serious amount of time on it.
    In the meantime I'll also try some other recovery apps to see if they have better luck...in case SanDisk's utility is not doing the job it's supposed to. I recommend others try the same so we don't unnecessarily reinvent the wheel 🙂
  24. Like
    horshack reacted to docmoore in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    Might give IsoBuster a try ... I have recovered a ton of stuff when the FAT was completely corrupted ... mainly from DVD/CDs
    It will dump any information it finds and is usually able to tell the type of file from the structure.
  25. Thanks
    horshack got a reaction from Emanuel in Canon EOS R5 so-called overheat timer defeated by a single screw in battery door   
    I looked into what it might take to recover the lost/missing video file after a battery pull. I reproduced the R5 hack scenario on my Canon RP by recording a minute of video on a freshly-formatted SD card and then pulling the battery while the door-sensor was inhibited. Like what Andrew discovered on his R5 the file is not corrupt but actually completely missing, both when attempting to play the video on the camera and when mounting the card on a computer. I then tried running SanDisk's RescuePro Deluxe and after scanning the entire card it didn't find the video.
    Here's what I think is happening. Obviously Canon has to write the video data to the card while recording, since it has no other place to retain it beyond the internal SDRAM buffer capacity. You can see easy evidence of this by watching the card access light continuously flicker. The strategy Canon is likely employing is to defer the writing of the official FAT metadata until the recording is orderly stopped. This would include elements such as the FAT directory entry, which anchors the file, and possibly also the linked FAT allocation tables. This would explain why RescuePro couldn't find the file, since there isn't enough or any recognizable metadata to reconstruct the orphaned data sectors associated with the video. Canon is either caching that information in memory until the file is closed or is writing it somewhere on the card that isn't linked to the official FAT structures. The benefit of this strategy, assuming they're using it, is that it prevents any potential filesystem-level corruption from incomplete metadata updates, since none of the official metadata structures are updated to link to the file until it's closed. The downside of this strategy is that it orphans all the data from the file and makes recovery more complicated.
    There are a few strategies in devising a recovery app for this situation. The first would be to reverse-engineer exactly what if any metadata Canon is writing during the recording and to where. If that orphaned metadata is in the same format as actual FAT structures then it should be relatively straightforward to create a placeholder directory entry and link to it. If the format of that metadata is proprietary to Canon then the process of reverse-engineering its structure would be much more complex. This would all be done by using a sector editor and block-based search tools to compare a freshly-formatted card to one which has a missing video after recording. And lots of effort.
    I was actually an embedded firmware storage engineer for most of my professional career so this would be in my wheelhouse. I'll see how much demand there might be for this before I spent any serious amount of time on it.
    In the meantime I'll also try some other recovery apps to see if they have better luck...in case SanDisk's utility is not doing the job it's supposed to. I recommend others try the same so we don't unnecessarily reinvent the wheel 🙂
×
×
  • Create New...