horshack

40 minutes ago, Electroholic Anonymous said:

72C is about the temperature that my 2011 Macbook Pro 17 inch idles in, because Apple back then did not want to annoy users with whining fans, it correctly thought that some 70C is cold enough for a CPU, and only really kick up the fan speeds when approaching 100C, below 80C the fans are already coming down, and, just as I like it. What a hassle it was to make my Alienware laptop behave the same. Let's see whether I still have a working R5 this evening, or next month, or next year, a few years from now. Is it really the cripple hammer v2? Or is it a crude and hurried attempt at temperature and heat damage control by the magnificent engineers of Canon?

Keep in mind there are other components inside the R5 that may have thermal envelopes below DIGIC and Image Sensor, the most obvious of which is the media cards.  ProGrade list the maximum ambient operating temperature of their cards at 70C, beyond which they can start to throttle. If you look at the photos of the Chinese R5 teardown the CFE/SD bay sits right next to DIGIC and SDRAM. 

https://progradedigital.com/wp-content/uploads/2019/03/ProGrade_DS_CFexpress_Final_E.pdf

19 minutes ago, Electroholic Anonymous said:

Indeed. Also, every time a second start of the video did not give a card slow error, so only the first attempt writing to a power dropped SD or CFexpress card gives this behaviour.

Here are the prelimnary results of the 40 minute 8K IPB FAT32 test. From 15 to some 40 minutes in to the test. Things appear to stabilize around 72C. Let's see how it fares tonight.

Thanks. Things are getting pretty toasty at around 72C 😀 That also happens to be the temp that Hoka Key's test stopped after 69 minutes of 8K recording for the overnight-fridge test. Here's my graph of that test (on right - left is non-fridge test):

5 minutes ago, Electroholic Anonymous said:

Indeed, that is foreseen, however, in all my tests, it only appeared when not formatting a card after a power drop, and this single time with the SD it was me deleting the files on the PC, without formatting it, in the other 6 instances in which I formatted the SD card it did NOT happen. I never ever had a slow card error in normal use for over a month with the R5. I am not ruling out that the R5 could do that as you say, it's just that I did not experience it after much abuse.

Good, thanks. Dropping power to NAND devices while writing isn't good for their health, esp low-end consumer devices like SD cards that may not have good power-loss recovery mechanisms. They may interpret the CRC errors from those blocks on subsequent accesses as bad blocks and start to remap them, which can affect performance. To help avoid this I would try to time the battery pulls to occur in between card writes by monitoring the card-access light on the camera, presuming there are idle periods visible in between writes for the data rate of the video quality you're shooting.

20 minutes ago, Electroholic Anonymous said:

My external battery has cut the test short unfortunately, 40 minutes into the test. The R5 gave the message to charge or replace the battery. I will repeat the test that I posted on youtube, tonight, running for over an hour this time, using two Sandisk 128GB SD cards in FAT32. Showing the clock again, and connecting the Ninja for the low-res 😉 4K Prores backup, and the thermal camera, and copying over all MP4 files in between, for proof. Just now I did not have time to copy the first run of MP4 files of the card. So tonight, or this afternoon if I have time.

Great, thanks. Please keep an eye out for those "slow card" error messages. One of theories about Canon's thermal management is to avoid overheating the media cards and causing them to throttle, esp. CFE but perhaps also SD. Once the cards get above a certain temp they start to internally throttle their performance to reduce heat, and this may manifest as those "slow card" error messages.

3 minutes ago, Electroholic Anonymous said:

Still working! Reformatted card does not give card slow error, on the start of the third 15 minute run which I started 4 minutes ago. The Adata microSD came out with 64C (degrees Celsius). The card slot and a nice 51 C, the R5's back at the location of the Digic stays at a stable and healthy 37C, it does not have a fever yet, and breaks no sweat.

Thanks. Here's the batch file I wrote that will extract the EXIF temps and put them into a .CSV file. Pass it the directory where you put all the .MP4s and the final still-image, plus a *.* mask. For example: get_exif.cmd c:\pics\*.*

@echo off
REM Put results.csv in same directory as files we're processing
SET resultsfile="%~dp1results.csv"
echo %resultsfile%
exiftool.exe -q -m -ext MP4 -ext JPG -ext CRW -ext CR2 -ext CR3 -p "${CreateDate;$_=(split(' ',$_))[1]},${FileName},${Duration#},${CameraTemperature#}" %1 > %resultsfile%

Above a certain filesize exiftool complains about not having large file support. Hopefully that wont be the case for the split-4GB files.

11 minutes ago, horshack said:

Fantastic, that's a working solution for SD then. Canon might have removed the legacy FAT32 support starting with CFE since it's an easy technology inflection point to do so. 

Could I ask a big favor? Could you possibly run 8K recording for an hour, using battery pulls as necessary? Also, after you reach the hour could you immediate take a photo? I'll post a batch file that will extract the EXIF temperatures from all the files and put them into a .CSV file I can graph with Excel. Typing on my phone right now so don't have access to my computer this moment. Thanks again!

The data rate for 8K is about 162 MB/s so you may need to copy the files halfway through so you don't run out of card space. 

Oops, that 162 MB/s is for 8K ALL-I. Using IBP it should be about 81 MB/s

10 minutes ago, Electroholic Anonymous said:

After a 6 minute run just now, it gives me 15 minutes again. As I am writing this the R5 is recording a 15 minute run. Will report back in 15 minutes 😉 No, I have two 1TB Integral Ultima CFE cards, and one Sandisk 512GB card. The Integrals run colder by the way. No of those 3 gave me an error or other disappointing behaviour, for a month full of abuse and power interruptions, hats off.

Fantastic, that's a working solution for SD then. Canon might have removed the legacy FAT32 support starting with CFE since it's an easy technology inflection point to do so. 

Could I ask a big favor? Could you possibly run 8K recording for an hour, using battery pulls as necessary? Also, after you reach the hour could you immediate take a photo? I'll post a batch file that will extract the EXIF temperatures from all the files and put them into a .CSV file I can graph with Excel. Typing on my phone right now so don't have access to my computer this moment. Thanks again!

The data rate for 8K is about 162 MB/s so you may need to copy the files halfway through so you don't run out of card space. 

2 hours ago, Electroholic Anonymous said:

IT WORKS!!!

This is with a Sony UHS-II card, R5 recording 8K 23.98p IPB in roughly 680Mbps. I will try some other modes next, but I trust this will work in any mode available on SD. Pity that RAW does not work with this neatest of tricks. You guys are geniuses!

 

Thanks! Did you verify that the thermal timer didn't reset after pulling the battery for this FAT32 case?

Interesting how the CFE didn't work. Hoka Key on the dpreview forums wasn't able to get his 320GB CFE card recognized by his R5 either when formatted to FAT32. I wonder if Canon just disabled FAT32 support for CFE. Do you have a smaller CFE card to try by any chance?

16 minutes ago, visionrouge said:

It's always laike this with all Canon camera. If you lose the power, all parameters are not saved.

But here is what you should do.
Put your camera parameters as you like until you are ready to shoot.
Turn off the camera with the power switch (The recording parameters as shutter speed, aperture, iso,...) are saved there
Turn on the camera.
Record and drop the battery when you like.
When you put again the battery, the parameters will be the same as last turn on/off sequence.
 

The goal isn't to save the parameters...the goal is to *not* save them 😀 It's the entire basis of the workaround for the timer element of the R5's thermal algorithm.

Btw, you don't have to turn off the camera to save the parameters. Taking a photo or a video or moving the mode dial between stills/movie mode saves them as well.

18 hours ago, Electroholic Anonymous said:

Otherwise, if I find a way to recover the DAT file into a healthy CRM file, I might just use that. It should not be difficult, because ending the 17 min recording in my video, does not appear to take any considerable processing. If I compare the DAT file to the CRM file with a hex editor (HxD),  the data looks the same (also in SciPy and Matlab I get the same data entropy readings, of course, it's compressed RAW data after all) The header is different though, what's missing is metadata, how many frames are recorded, block entries for audio and video data, etc. A simple copy over from the header of the healthy CRM to the unfinished DAT file, did not work well in Premiere and Resolve, they recognize it, but Premiere gives frame access exceptions (I forgot the precise error description), and Resolve shows black video. It is not easy (for me), to rebuild the correct metadata for the DAT file. It is not easy for the c200 either because it lasts a long time on that camera to repair a DAT file. Hopefully our friends at Grau expand their program with support for the R5 soon, it support a long list of camera's and formats, I trust they will be able to do it. Their program is able to repair the MP4 files, but the resulting 8K video is all corrupted, although the metadata is correct, correct codec information, number of frames and all. I have send them a healthy and corrupted MP4 too.

 

@Jn- came up with an alternate idea - use FAT32 on the card and the camera will automatically split up the files into 4GB. I don't have an R5 but I tested this on my RP and verified it doesn't save the NVRAM settings for each of the 4GB file splits. You can read more about it in the following two links. I've asked Andrew and J. Marcus to try this on their R5 but I don't think they've had a chance yet.

Prior to that I did some work on recovering the .DAT file but stopped after the above solution looked like a better alternative. You can read about the work I did in the following link:

3 hours ago, horshack said:

Just tried it for a longer video, this time 10:00, to test two splits and it still works. Two perfectly-good .MP4 files, each approx 4GB and 4:41 in length, and an RP with amnesia after powering it back on.

The only limitation of this technique is 32GB, since Canon cameras format all cards > 32GB as exFAT, even though FAT32 supports volumes up to 2TB. But there's a workaround for that as well - format the >32GB card on computer instead. Windows 7/10 wont let you format cards > 32GB to FAT32 but there are third-party utilities that will. I just used http://www.ridgecrop.demon.co.uk/index.htm?guiformat.htm on a 64GB card and the camera accepted the card with no complaints. Recording to it right now on my RP to test splits for a very long video. And while Windows wont let you format >32GB cards to FAT32 it will mount them just fine.

Update:

• I recorded 25:00 of video then battery pull with no issue on my RP. All ~4GB MP4 segments were fine, with naturally the last one missing
• Don't let the recording reach the 29:59 legacy limit - if you do the camera will save the NVRAM settings when it automatically ends the recording
• The Ridgecrop formatter tool link I posted for Windows only worked once for me. After that the camera wouldn't accept any cards formatted with it. I then tried the formatter built into OSX using the FAT32 and MBR options (not GPT) - the camera accepts those cards but then displays an error whenever you take a photo or end a video recording. The only reliable and consistent way I've found to format > 32GB cards for FAT32 that my RP would accept and work with is under Linux. You can see instructions how to do so here.
• Here's a screenshot of the ~4GB segements on a 64GB SD card formatted to FAT32 for my 25:00 recording:

29 minutes ago, horshack said:

You sir win the internet today. I just tried it on my Canon RP, which like the R5 forgets any changes to the camera's settings (like exposure settings) if you pull the battery in the middle of the recording but will remember the changed settings if you stop a recording normally.

Procedure:

1. Tape down the door sensor and leave the door open so the battery is accessible
2. Put a 32GB card in the camera and
3. Turn the camera on and format the card. It will automatically use FAT32 cards with a capacity <= 32GB
4. Start a video recording, in this case I used 4K 24P, which has an average total video/audio bitrate of 14.43 MB/s.
5. Shoot a video whose length will exceed 4GB. For my 4K 24P that's any video longer than just under 5:00. I shot one for exactly 5:00
6. Pull the battery
7. Check the card and verify there's a single .MP4 file
8. Reinsert the battery and verify the camera "forgot" the previous session's settings

We know the camera saves its settings whenever it has successfully recorded a video. The question is whether @Jn- idea will avoid that save for the FAT32-case of splitting up a video file at 4GB/each (FAT32 limit), provided we still pull the battery at the end of the recording. In other words, will the camera save NVRAM for each intermediate 4GB-split file or only for the final file after properly stopping a recording?

Drumroll...It works! My RP's aperture was f/4 when I turned it on for step #3. I changed it to f/11 for the test. After reinserting the battery my RP's aperture reverted back to f/4!

Andrew, please test this ASAP on your R5 and see if it matches my RP's behavior of forgetting the settings, the most important of which is of course the thermal timer.

 

Just tried it for a longer video, this time 10:00, to test two splits and it still works. Two perfectly-good .MP4 files, each approx 4GB and 4:41 in length, and an RP with amnesia after powering it back on.

The only limitation of this technique is 32GB, since Canon cameras format all cards > 32GB as exFAT, even though FAT32 supports volumes up to 2TB. But there's a workaround for that as well - format the >32GB card on computer instead. Windows 7/10 wont let you format cards > 32GB to FAT32 but there are third-party utilities that will. I just used http://www.ridgecrop.demon.co.uk/index.htm?guiformat.htm on a 64GB card and the camera accepted the card with no complaints. Recording to it right now on my RP to test splits for a very long video. And while Windows wont let you format >32GB cards to FAT32 it will mount them just fine.

1 hour ago, Jn- said:

Not addressing the file corruption directly, but if the card is formatted in fat32, all of the smaller 4gb video clips recorded and saved except the last one, at battery eject should be fine.

You sir win the internet today. I just tried it on my Canon RP, which like the R5 forgets any changes to the camera's settings (like exposure settings) if you pull the battery in the middle of the recording but will remember the changed settings if you stop a recording normally.

Procedure:

1. Tape down the door sensor and leave the door open so the battery is accessible
2. Put a 32GB card in the camera and
3. Turn the camera on and format the card. It will automatically use FAT32 cards with a capacity <= 32GB
4. Start a video recording, in this case I used 4K 24P, which has an average total video/audio bitrate of 14.43 MB/s.
5. Shoot a video whose length will exceed 4GB. For my 4K 24P that's any video longer than just under 5:00. I shot one for exactly 5:00
6. Pull the battery
7. Check the card and verify there's a single .MP4 file
8. Reinsert the battery and verify the camera "forgot" the previous session's settings

We know the camera saves its settings whenever it has successfully recorded a video. The question is whether @Jn- idea will avoid that save for the FAT32-case of splitting up a video file at 4GB/each (FAT32 limit), provided we still pull the battery at the end of the recording. In other words, will the camera save NVRAM for each intermediate 4GB-split file or only for the final file after properly stopping a recording?

Drumroll...It works! My RP's aperture was f/4 when I turned it on for step #3. I changed it to f/11 for the test. After reinserting the battery my RP's aperture reverted back to f/4!

Andrew, please test this ASAP on your R5 and see if it matches my RP's behavior of forgetting the settings, the most important of which is of course the thermal timer.

52 minutes ago, ajay said:

The question is will some of the heavy-hitters out on YT and the web sites will call Canon out on this? They should if they're not biased. It's nice to see most who have already started the drum beat are crediting Andrew as well as others. Well deserved.

Matt Granger posted a video yesterday:

45 minutes ago, Andrew Reid said:

Sure will do. I'll send you these via Wetransfer soon - do you want 10bit HEVC or 8bit H.264 - or both?

All-I for the pair of videos. thanks. I think 30 seconds might be too big to share - 15 seconds/each should be plenty.

1 hour ago, Andrew Reid said:

@horshack

Are you working with the 8K MP4 DAT files, or 4K HQ truncated recordings too? I can send you a few different flavours of file if this helps, might be interesting to compare the different codecs.

I also need to test 4K HQ with the battery pull after starting and stopping recordings prior to one last sacrificial short clip to facilitate the battery pull during recording, and seeing what the camera remembers from the previous 4K HQ clip record times in NVRAM and what it doesn't. Card swap is also key to do.

I'm working with 4K H.264 files from a 1DX that @docmoore provided me. I can actually use a pair of R5 ALL-I 4K files - perhaps those will be easier to reconstruct. Can you shoot two 4K ALL-I videos for me, each 30 seconds in length, one that is recorded normally and the other where you pull the battery after 30 seconds. The first will be a regular .MP4 and the second will be a .DAT file. Then put them up on a share I can download from? Please have some kind of sound playing while recording - that will help me in the debugging effort find and sync up video/audio frames. Thanks!

3 hours ago, docmoore said:

You have made great progress ... I wonder how specific the MP4 header is and whether is has a similar temp file that is converted to the final MP4 header when the file is finalized. I assume it holds the key to interpretation of the file by the NLE 

Fun to see this unfold 

I think I've found out why the video playback is blank and the there's no audio. The MP4 header has 'stsz' and 'stco' atoms that are indexes into the actual video and audio data in the mdat atom. Those atoms are used by players/NLEs to correlate time to video/audio frames. Since I copied the entire MP4 header from the "good" video those indexes will be wrong for the "bad" video and have to be rebuilt. I found an open-source tool named uncut that does this (https://github.com/ponchio/untrunc), documentation here. Built and ran it but it wasn't able to find any of the frames in the files. Will be looking into the code to see if it can be tweaked to work on these files

Petapixel has picked it up:

https://petapixel.com/2020/08/24/simple-hack-proves-canon-eos-r5-overheating-limit-is-artificial/

9 hours ago, docmoore said:

Check messages ...

Made a lot of progress on understanding the .DAT files. Using my Canon RP as a test bed, I recorded two 1:00 videos, one regularly and the other power-interrupted. I then copied the entire SD card for each case as a file image to my system for analysis. Prior to recording video each SD was prepared by fully wiping it with dd, then formatting in the camera. I also went back to FAT32 because it's easier to analyze those structures than exFAT.

This yields two very similar filesystem images. For the regularly-recorded video SD image I see a single data file (MVI_0478.MP4 in this specific case). However if I analyze the directory entries in the FAT32 structure I see a second file that's been deleted, named ?VI_047.DAT. The '?' is there because the file has been deleted (0xE5 is the FAT32 marker to indicate a deleted directory entry), meaning its original name is MVI_0478.DAT. Even though MVI_0478.DAT has been deleted its FAT32 cluster location and most-recent file size is still available in the directory entry.

Looking at the contents of the .DAT I see an "mdat" field. This is the raw MP4 data holding the video/data of the video - it's called an "mdat atom" in MP4 container parlance. The size of the .DAT file is exactly 128KB less than the size of the .MP4. What's more, the on-SD location of the .DAT is exactly 128KB after the .MP4, which means the two files are actually contiguous with each other.

So the camera wrote the .DAT first (while recording the video) but left 128KB before it pre-allocated in the FAT32 cluster table. When the video recording stopped it then wrote the 128KB before the .DAT, which contains all the other MP4 atoms necessary to represent the file, including the 'moov atom' which was reported missing when trying to recover the .DAT as .MP4 earlier using open-source recovery tools (failed because it's missing the full 128KB .MP4 header).

The 128KB MP4 container header stuff + mdat atom content represents the full MP4 file.

So the .DAT files we're seeing for the interrupted recordings on the R5 are the actual raw video/audio data sans the proper 128KB MP4 header area.

On my Canon RP the .DAT directory entry is missing for the interrupt recording. However if I go to the location on the interrupted-SD image as where the .DAT file exists on the good-SD image I see the mdat data. So it appears that on my RP the camera writes to this area during recording without first setting the directory entry to reflect that area is allocated. If you're able to see the .DAT files from your 1DX that means the behavior of that camera is different. Not sure how the R5 acts. Either way, recovering the .DAT file for my RP is trivial because I know what cluster it begins on by comparing it to the good-SD image.

I found the .DAT area on the interrupted-SD image and spliced in the 128KB MP4 header from the .MP4 from the good image. The file plays...and I hear a quick snippet of sound, but no video. So there is work to do there. But at least it demonstrates the basic integrity of the full MP4 is in place.

To get this working on R5 images we'd have to determine how to build the proper 128KB MP4 container header, the same as how the camera builds it when the user ends a video recording. This might be as simple as splicing from a "good" .MP4 image, but probably requires a lot more effort, and then graft that 128KB header data with the the .DAT file to reconstruct the full .MP4

42 minutes ago, docmoore said:

Check messages ...

Thanks for the files. Tried two separate automated MP4 repair programs/scripts, both of which support reading the headers/metdata from the "good" file and attempt to graft that data over those missing elements in the bad file, specifically the missing moov atom in the corrupt file. No joy so far. Oddly when I examine the "good" file the moov atom is actually at the beginning of the file - I expected it to be at the end since that's what's missing from the corrupt file. It might be that Canon relocates the moov atom to the front after a successful recording end. It's usually placed at the beginning of files to facilitate better streaming as the file is being downloaded. Or it might be that IsoBuster didn't recover the file correctly. I'll keep digging.

Here's the Atom data from the good file, obtained from AtomicParsley <filename> -T

Atom ftyp @ 0 of size: 28, ends @ 28
Atom moov @ 28 of size: 262116, ends @ 262144
     Atom uuid=85c0b687-820f-11e0-8111-f4ce462b6a48 @ 36 of size: 65750, ends @ 65786
     Atom udta @ 65786 of size: 2056, ends @ 67842
         Atom manu @ 65794 of size: 20, ends @ 65814                     ~
         Atom modl @ 65814 of size: 38, ends @ 65852                     ~
         Atom urat @ 65852 of size: 16, ends @ 65868                     ~
         Atom free @ 65868 of size: 1974, ends @ 67842
     Atom mvhd @ 67842 of size: 108, ends @ 67950
     Atom trak @ 67950 of size: 16328, ends @ 84278
         Atom tkhd @ 67958 of size: 92, ends @ 68050
         Atom edts @ 68050 of size: 36, ends @ 68086
             Atom elst @ 68058 of size: 28, ends @ 68086
         Atom mdia @ 68086 of size: 16192, ends @ 84278
             Atom mdhd @ 68094 of size: 32, ends @ 68126
             Atom hdlr @ 68126 of size: 33, ends @ 68159
             Atom minf @ 68159 of size: 16119, ends @ 84278
                 Atom vmhd @ 68167 of size: 20, ends @ 68187
                 Atom dinf @ 68187 of size: 36, ends @ 68223
                     Atom dref @ 68195 of size: 28, ends @ 68223
                 Atom stbl @ 68223 of size: 16055, ends @ 84278
                     Atom stsd @ 68231 of size: 311, ends @ 68542
                         Atom hvc1 @ 68247 of size: 295, ends @ 68542             ~
                     Atom stts @ 68542 of size: 24, ends @ 68566
                     Atom stss @ 68566 of size: 296, ends @ 68862
                     Atom ctts @ 68862 of size: 4472, ends @ 73334
                     Atom stsc @ 73334 of size: 28, ends @ 73362
                     Atom stsz @ 73362 of size: 3364, ends @ 76726
                     Atom co64 @ 76726 of size: 6704, ends @ 83430
                     Atom sdtp @ 83430 of size: 848, ends @ 84278
     Atom trak @ 84278 of size: 8276, ends @ 92554
         Atom tkhd @ 84286 of size: 92, ends @ 84378
         Atom edts @ 84378 of size: 36, ends @ 84414
             Atom elst @ 84386 of size: 28, ends @ 84414
         Atom mdia @ 84414 of size: 8140, ends @ 92554
             Atom mdhd @ 84422 of size: 32, ends @ 84454
             Atom hdlr @ 84454 of size: 33, ends @ 84487
             Atom minf @ 84487 of size: 8067, ends @ 92554
                 Atom smhd @ 84495 of size: 16, ends @ 84511
                 Atom dinf @ 84511 of size: 36, ends @ 84547
                     Atom dref @ 84519 of size: 28, ends @ 84547
                 Atom stbl @ 84547 of size: 8007, ends @ 92554
                     Atom stsd @ 84555 of size: 103, ends @ 84658
                         Atom mp4a @ 84571 of size: 87, ends @ 84658
                             Atom esds @ 84607 of size: 51, ends @ 84658
                     Atom stts @ 84658 of size: 24, ends @ 84682
                     Atom stsc @ 84682 of size: 748, ends @ 85430
                     Atom stsz @ 85430 of size: 6548, ends @ 91978
                     Atom co64 @ 91978 of size: 576, ends @ 92554
     Atom trak @ 92554 of size: 947, ends @ 93501
         Atom tkhd @ 92562 of size: 92, ends @ 92654
         Atom mdia @ 92654 of size: 847, ends @ 93501
             Atom mdhd @ 92662 of size: 32, ends @ 92694
             Atom hdlr @ 92694 of size: 33, ends @ 92727
             Atom minf @ 92727 of size: 774, ends @ 93501
                 Atom nmhd @ 92735 of size: 12, ends @ 92747
                 Atom dinf @ 92747 of size: 36, ends @ 92783
                     Atom dref @ 92755 of size: 28, ends @ 92783
                 Atom stbl @ 92783 of size: 718, ends @ 93501
                     Atom stsd @ 92791 of size: 50, ends @ 92841
                         Atom tmcd @ 92807 of size: 34, ends @ 92841
                     Atom stts @ 92841 of size: 24, ends @ 92865
                     Atom stsc @ 92865 of size: 40, ends @ 92905
                     Atom stsz @ 92905 of size: 20, ends @ 92925
                     Atom co64 @ 92925 of size: 576, ends @ 93501
     Atom free @ 93501 of size: 168643, ends @ 262144
Atom mdat @ 262144 of size: 730194240 (^), ends @ 0
             (^)denotes a 64-bit atom length

 ~ denotes an unknown atom
------------------------------------------------------
Total size: 730456384 bytes; 66 atoms total. AtomicParsley version: 0.9.0 (utf16)
Media data: 730194240 bytes; 262144 bytes all other atoms (0.036% atom overhead).
Total free atom space: 170617 bytes; 0.023% waste. Padding available: 1974 bytes.
------------------------------------------------------

3 minutes ago, docmoore said:

Easily done but would you rather have a file that is not Canon Raw ... their Canon Raw Developer and Davinci Resolve ... I assume FCP ... are needed ....

I can record clips in 4K DCI all-I @ 23.976 if that is better for you.

Oops, forgot you mentioned raw. I definitely want to focus on processed first. Can you shoot a pair of H.264 4K 24P videos? I'll work on H.264 first then consider H.265 and/or ALL-I

11 minutes ago, docmoore said:

Very short recording ... only 1.38GB but that appears about right for the 5.5K RAW it was recording. I believe you are correct ... the question is how to add the container and whether the EXIF data can be written.

Good. The EXIF can always be grafted with exiftool on but it's probably superfluous for desktop NLE use. Can you possibly share the file via Dropbox or Google Drive? Could you also take another video of the same length that's recorded normally and share that with me as well? I'll be generating my own files too but the process on wiping the cards to perform each experiment is slowing me down so I have free time in between each.

1 minute ago, docmoore said:

Just ran a short clip on my 1DXIII pulled the battery and extracted the DAT file to my desktop. ISObuster attempted to convert it to mpg but the resulting file

was unplayable in VLC, unrecognized by FF:Works and Davinci Resolve Studio. The conversion in ISObuster was at 2.1MB/s ... so even if it were readable the length of time

to convert would be untenable.

Thanks. That probably saves me from buying ISObuster for this effort. Was the size of the file about what it should've been for how long the video was? If so then the file may have all the salvageable data recovered and just needs to have the tail of the MP4 container repaired as discussed earlier.

 I haven't been getting the .DAT but just realized it's because I'm using FAT32 whereas everyone else is likely using exFAT. It defaulted to FAT32 due to the smaller card I was using (32GB), which I chose to make the imaging/recovery experiments faster. If I switch the same card manually to exFAT then the .DAT file starts showing up after an interrupted recording on my Canon RP. I'll be focusing on exFAT in my ongoing work since that's what most are probably using.

1 hour ago, ajay said:

I don't understand why you would go through all that work to reconstruct the file. Why not record what you want, then record another short, bogus file and kill the battery. Forget about trying to reconstruct the file and just create a dummy file that you don't care about.

It's definitely worth a shot but the evidence suggests that stopping a recording causes the current thermal management state to be written to NVRAM at that point. This is likely why J Marcus only got 5:00 back when pulling the battery - that's because the last NVRAM update was from the previous power-on session where the last video file was saved and the camera reported 5:00 available. So pulling the battery thereafter reverts back to whatever was last saved in NVRAM, which for him was 5:00

1 hour ago, docmoore said:

It is possible the the MP4 is an older file ... if a low level format of the card was not done ... it may still appear if deleted and not overwritten by a subsequent

write operation. I assume your file is the .mhk

It was with an in-camera low-level format, which doesn't actually wipe the entire card because it completes in just a few seconds. After I posted earlier I did another experiment where I completely wiped the card with 'dd' under Linux, then did an in-camera low-level format and repeated the test video w/battery pull. After that IsoBuster only found a single file, named Recovered File 1. mp3 with a size equal to the full capacity of the SD. Which means it's actually less encouraging than before, at least the idea of using an off-the-shelf program rather than engineering a custom solution.