Jump to content

Effective Storage Handling with Final Cut Pro X


Wit
 Share

Recommended Posts

I'm testing the most optimum storage handling method for extended shoots. Being on the road with minimum back up means. Here's what I found so far - and what questions are left. Would help if you gave your thoughts on this ...

 

 

Set Up:

 

- Macbook pro SSD 1TB

- 2 x LaCie rugged mini 1 TB

 

Storage method:

 

- Edit on Macbook

- Store project file on Macbook drive under Films - Final Cut Projects 

- Shoot more compressed AVCHD files

- Store full copy of SD card in xtra folder in the dedicated Event folder in Final Cut Events

- Run one Time machine back up copy on LaCie 1 disk

- Make copies of the Time machine back up on LaCie 2 disk 

 

 

My idea is to store and edit until my notebook will be full. As soon as my notebook is full, I will empty the Original Media folder to safe space. I will restore the data from the back up disk to the Original Media folder when needed as soon as I'm in the situation to do. Tested this method and it works. Of course an accurate administration system will help. 

 

The other method will be to import the data again from the copy of the SD card. I didn't test this method yet but I figure it will be time consuming as I somehow think the reconnecting has to be done file by file.

 

 

My questions are:

 

1. Can I run an automatic back up via Time Machine on back up disk 2 too or will this confuse the system? 

    Therefore better refrain to making manual copies whenever there has been changes.

2. I guess the data on the backup disk will be compressed. Won't this harm the quality of my footage? 

3. What reconnecting methods are best? Via Time machine or reconnecting via SD card?

4. General thoughts on my storage handling as described above are welcome!

 

 

Thanxs!

Link to comment
Share on other sites

With FCP X, there is almost no 'wrong' way. You seem to be quite conscious about what you do (because one can lose data!), so I just share a few thoughts:

• How much redundancy do you need? How many instances of backups are sane, and when exactly begins paranoia?

• Do you want the render files to be included in the Timemachine backup? 

 

A Timemachine backup implies that your system's harddrive, your SSD, may fail. Whereas this is possible, a whole different scenario is more likely to happen: Bug infection. Not death, but a condition where you beg for death. You have, for example, some version of FCP X. You are in the midst of editing some projects (which were sequences in older NLEs), and you install a plugin. All of a sudden, FCP X displays the beachball, renders random green or purple frames. You didn't make a Timemachine backup, because you felt secure with an expensive Raid :( . You trash preferences (old first aid remedy), no success, you uninstall the plugin, no success. Hell, the way to restore your project is full of pain. You can better go back to start.

 

Timemachine is your rescue not only with third-party plugins (also, beware, Quicktime-plugins, like Perian), I also don't trust a new version of FCP X. Keep your old versions as backups!

 

I make a Timemachine backup at the end of the day. 

Additionally, I have a freshly installed system-backup on another drive.

And the camera archives on another (the media are on three drives that way!)

 

Use the import-tools of FCPX to filter your usable clips, it helps a lot to save space!

Link to comment
Share on other sites

I'm not paranoia - it's what other traveling filmers/photographers advised me - 2 back ups. 

Divided among two travel bags of two separate persons. They also recommend sending material home but I'm not sure about that. I just wanna understand what I'm doing so I don't loose material because of ignorance.

 

I don't know if I want the rendered files on my back up - do I are they taking up that much space? If not how can I excluded them?

Yeah for sure I won't update the system while using this method.

 



 


I make a Timemachine backup at the end of the day. 

Additionally, I have a freshly installed system-backup on another drive.

And the camera archives on another (the media are on three drives that way!)

 

What do you mean with the second phrase? Unplug the first harddrive and plug in the second to make again a Time Machine back up. Wo'nt this mess your back up system?  

Link to comment
Share on other sites

I somehow skipped the part where you wrote that you were travelling. Then of course your care is appropriate.

 

I don't know if I want the rendered files on my back up - do I are they taking up that much space? If not how can I excluded them?

 

With very old versions of the classic FC (and all older NLEs), when harddisc space was expensive and real time performance didn't deserve the name, you often judged an effect by how it looked on the current frame only (which was always rendered in full quality). You had to see things like an animation, and then you had to render, but you triggered it manually by selecting the clip(s) in the sequence and hitting ⌘+R. Render files can consume more space than footage, and none of them are actually used for final export, they are for preview purposes only. You can put off background rendering in FCP X's preferences (here), then you can render a selected clip by hitting CTRL+R. 

 

Assumed your project consists of a hundred animations with a dozen more filters on each. Then again, you wouldn't want to keep the render files. It would be wiser to export an entire self-contained movie as a reference and delete all render files. This will delete not only all rendered clips from the current state of your projects, but also those of all former states, because, as you know, FCP X works as it's own Timemachine, and since Lion, every tiny change on every file gets auto-saved. Imagine all the automatically rendered clips!

 

Render files not only eat disc space (which is cheap nowadays), they eat RAM. This is because every file that's used to play back a sequence needs to be kept in short time memory together with all files it refers to. With this fact in mind, think of the size of your events and projects. Even on a 32 GB RAM machine, ONE giant event pool with thousands of clips will eventually slow down the system, if you use clips from this giant event in multiple projects. The same, if you had a hundred events, each only with 20 - 50 clips, but one giant project of two hours, using clips from all events and additionally cross-referential compound clips! Don't get me wrong: You can do this, and you should do it, but not all the time and not with, say, 4 GB RAM. Use Event Manager X ($4,99), until a future version of FCP X allows to uncheck unused events and projects (the main advantages of the program are, it makes you aware of your media organisation and it cleans up your GUI!). It also 'ejects' unused volumes ...

 

Then there is GPU acceleration. If your graphic card allows it, you can see all effects in real time without rendering. You can - and should - always choose >preferences >playback >better performance over >high quality, when there is a problem with playing back unrendered clips. If you pause, the current frame will always be 'high quality'. 

 

I make a Timemachine backup at the end of the day. 

Additionally, I have a freshly installed system-backup on another drive.

And the camera archives on another (the media are on three drives that way!)

 

What do you mean with the second phrase? Unplug the first harddrive and plug in the second to make again a Time Machine back up. Wo'nt this mess your back up system?  

 

First line: Backup for the work.

Second line: Backup to reset the system, to format the HD.

Third line: If, for some reason I have to relink to original media. I wipe and re-use the SD-cards.

Link to comment
Share on other sites

Thanxs! Deleting rendered files sounds as a wise thing to do.

 

For other readers I'll include a video how to. But What do I do with the rendered files from projects which aren't anymore on my FCP X page? How do I delete them - which map or sub files should I remove?

 

 

 

 

 

 

It sounds like you're not advising me to store everything on the SSD of the notebook are you?

If you want - I can read German too.

Link to comment
Share on other sites

But What do I do with the rendered files from projects which aren't anymore on my FCP X page? How do I delete them - which map or sub files should I remove?

 

Why are the projects no longer on the surface? Did you move them? Moving (='hiding', as FCP X calls it in the manual) is a method to get them out of the way without deleting them. Move them back, restart FCP X, they will appear. Select them, delete files. I suppose you could always use the finder to get to the render files and put the whole folders into the trash. But I never tried this myself.

 

It sounds like you're not advising me to store everything on the SSD of the notebook are you?

 

Rather that than reading/writing anything over a USB connection. I suppose you didn't experience big problems with having the OS and program on the same volume as the clips?

 

Note, that with AVCHD you always work with a copy, even if you checked 'original media'. The copying process is so fast, you don't realize it, but in the 'original media' folder live H.264 copies of your '.mts' - clips (EDIT: In case you chose 'optimized media' - the default for AVCHD -, the files will be big, but edit-friendly ProRes-files, check this).

 

There are two advantages in that:

 

1. You can connect your card reader and a LaCie drive to the MacBook, open FCP X, hit 'import' and there 'create archive'. See here for details (This was obviously before the unified import of 10.0.6, the rest stays valid). In the opening dialogue, you choose the LaCie as volume where your SD-card should be archived. Now you wait, until the backup is complete. Then eject the card reader.

 

2. After that, you create a new event with your SDD as volume, hit import and choose the camera archive on the LaCie.

 

3. Here comes a very interesting feature only available with original SD-cards or camera archives:

Archiv.jpg

(here is your German  ;) )

 

The blue icon selected here is the 'clip view' of the import window. It allows for you to skim through all your clips at once, qualifying them very quickly (granted, were the archive not connected only via USB, you could skim waay faster!). Moreover, you can select multiple regions of one clip before you hit import.

 

4. That way, you only copy the portions of the original clips you actually plan to use in your project, saving space on your system volume!

Link to comment
Share on other sites

Why are the projects no longer on the surface? Did you move them? Moving (='hiding', as FCP X calls it in the manual) is a method to get them out of the way without deleting them. Move them back, restart FCP X, they will appear. Select them, delete files. I suppose you could always use the finder to get to the render files and put the whole folders into the trash. But I never tried this myself.

 

Because I threw them away. And I didn't make a back up as it where all trials. But I figure the rendering files are still existing?

Can I remove the all the folders from the map Films/Final Cut Events/Camera render files and

 the  from the Peaks data from the Render Files map with no harm?  Nothing important yet on this machine.

 

 

 

 

h. But I never tried this myself.

 

Rather that than reading/writing anything over a USB connection. I suppose you didn't experience big problems with having the OS and program on the same volume as the clips?

 

Note, that with AVCHD you always work with a copy, even if you checked 'original media'. The copying process is so fast, you don't realize it, but in the 'original media' folder live H.264 copies of your '.mts' - clips (EDIT: In case you chose 'optimized media' - the default for AVCHD -, the files will be big, but edit-friendly ProRes-files, check this).

 

There are two advantages in that:

 

1. You can connect your card reader and a LaCie drive to the MacBook, open FCP X, hit 'import' and there 'create archive'. See here for details (This was obviously before the unified import of 10.0.6, the rest stays valid). In the opening dialogue, you choose the LaCie as volume where your SD-card should be archived. Now you wait, until the backup is complete. Then eject the card reader.

 

2. After that, you create a new event with your SDD as volume, hit import and choose the camera archive on the LaCie.

 

3. Here comes a very interesting feature only available with original SD-cards or camera archives:

Archiv.jpg

(here is your German  ;) )

 

The blue icon selected here is the 'clip view' of the import window. It allows for you to skim through all your clips at once, qualifying them very quickly (granted, were the archive not connected only via USB, you could skim waay faster!). Moreover, you can select multiple regions of one clip before you hit import.

 

4. That way, you only copy the portions of the original clips you actually plan to use in your project, saving space on your system volume!

 

 

MMM it's too late now to dig into this but hey thanks I'll give it a go tomorrow. Actually I didn't do much editing on the macbook pro yet. So I don't know if I will encounter speed problems yet.

Link to comment
Share on other sites

Hello Blanche.

 

Are you sure about your Macbook Pro with a 1 TB SSD?  I ask because until now Apple sells Macbooks  with only 256/512 or 768GB SSD.

 

I see in your scenario some problems:

In order that Time Machine can keep backups over several weeks/month your Time machine disk should have the capacity of 2x, at least 1.5x of your Macbook hard disk.

When your notebook is full, your Time Machine disk will also be almost full and Time Machine will begin to delete the first backups you made, that means your are going to lose data.

 

Because you are very concerned about the security of the data on your LaCie disks, you are planning to have two of them for backup. But what are you going to do when your Macbook drive fails? Are you prepared?

 

OK, what would I do in your situation?

I would store all my new events and projects on my Macbook and everyday I would run Time Machine (on the first LaCie) in order to backup those data.

All finished projects (including events) I would move to an external drive (second LaCie) in order to make space. At the end of the day I would backup this external drive to another external drive (third LaCie). And I would buy another 512GB SSD for the case that my Macbook SSD fails.

 

Let’s check this scenario:

When your Macbook SSD fails, you have a copy on the Time Machine. You can install the SSD in reserve and re-install everything from the Time Machine copy.

When your Time Machine drive fails, you have still the original data on the Macbook.

When one of your external drives fails, you still have a copy on the other external drive.

 

It seems, in this scenario your data are pretty safe. What do you need? 3 x LaCie and a reserve SSD for your Macbook.

 

BTW, Time Machine does not compress the data.

 

Greetings.

Link to comment
Share on other sites

Thankx! I was so wrong only have 250,14 GB on SSD  :o - really thought it was more. MMM this asks for another workflow maybe.

Yes if my macbook got robbed or is broken I have a problem but I can't fix all uncertancies in live.

 

Thank you so much for your answers - will dig into this tonight!

Link to comment
Share on other sites

  • 1 year later...

4. General thoughts on my storage handling as described above are welcome!

 

Well, one thought that came to my mind was about this:

 

 

- Edit on Macbook

- Store project file on Macbook drive under Films - Final Cut Projects 

- Shoot more compressed AVCHD files

- Store full copy of SD card in xtra folder in the dedicated Event folder in Final Cut Events

- Run one Time machine back up copy on LaCie 1 disk

- Make copies of the Time machine back up on LaCie 2 disk 

 

 

Maybe it's just me, but I would never edit on the MacHD. I'd let FCPX make the default folder(s) it wants to do in the main HD of the Mac, but never use them for anything.

I tell FCPX to make a new Projects and Events folders on a separate, relatively big and relatively fast external hard drive, and then import (copy) the sources and edit from there. I always keep the original source materials elsewhere on yet another hard drive, or import them into the work Projects folder directly from the SD(D) drives. When I'm done editing and finish the project, I consolidate the library and then get rid of the unnecessary files to save space.

 

There are also some FCPX library managers available. I've got a free/cheapo app called Final Cut Library Manager. It might be handy at some point.

 

I've never bothered with Time Machine when it comes to the render files and finished projects. It feels like an unnecessary step, as FCPX is making copies of your imported files anyway, if you let it do so, and when the render files are missing, FCPX will make new ones on the fly. So far I've done my backups manually onto a separate backup disk.

 

Just my general thoughts. Effective or not, that's how things are in the Quirky HQ at the moment. I might come up with a new workflow later next year, if I manage to gobble up  enough dough to upgrade my editing hardware.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

  • Similar Content

    • By pintowar
      2 SSD Crucial MX100 of 512GB. Used but in perfect working order. See real photos and health and performance tests. MLC memories, more durable and of higher quality than the current models of Crucial.
       
      Price: 280 €
       





    • By Kelly
      Well hello there EOSHD forum readers,
      I've just posted my short film, Axiomatic, to Vimeo.
      Shot with an older version of the 5D3 Raw hack without the audio component - remember when we had to reload the Raw modules each time after restarting the camera? Ah well...it still worked flawlessly and I cannot thank all the ML developers enough for what they unlocked. 5D3 raw is simply amazing.
      Spanish, French and English subtitles are available for whomever needs them and viewer discretion may be advised as there is brief nudity, violence and coarse language. However I tried to make everything as tasteful as possible.
      Hopefully you can take some time out of your busy day to view it and please feel free to share it with any of your friends or contacts that might enjoy some dark Canadian cinema.
      Kelly
      http://filmshortage.com/dailyshortpicks/axiomatic/
    • By Kelly
      Hello EOSHD forum 
      First time poster and long time reader here. I wrote/directed a short film based on Aussie sci-fi novelist Greg Egan's story 'Axiomatic'. I shot this on a 5DmkIII with Zeiss and Takumar primes and the older 1080p ML Raw hack as the 4k option wasn't available yet. Please feel free to check out the teaser trailer on Vimeo:
       
    • By Mickey Gaidos
      I shot this promotional video for the annual Tango Festival here in New Orleans.
      Shot with c100II and GH5, I really wanted to test out the improvements in the lowlight capabilities with the GH5 over shooting last years festival with the GH4, I was impressed!

      Edited in FCPX
       
    • By Naive Studio
      First online social media advert for J'adore Models Manchester, shot on the Sony A7SII in SLog3. Used the excellent Zeiss loxia primes, 50mm f2 and 35mm f2.
      Recorded full-frame 4K at 24fps, shutter lower than I would prefer due to room lighting restrictions but decided to roll with it instead of using LEDs to keep a cinematic look, pools of light and dark shadows etc. Looking to shoot more of these throughout the year and would love to know what peoples thoughts were...
      We are a boutique agency in Manchester. Our guys and girls have the freshest and coolest faces on the scene. We believe totally in presenting the next wave of unique, diverse and beautiful models to the industry and we’re about exceeding clients’ expectations, every time.
      jadoremodels.co.uk
      Filmed on the Sony A7SII
      Rig: Movcam A7SII cage
      Lenses: Zeiss Loxia 50mm f2 / Loxia 35mm f2
      4K / 1920 x 1080 HD / 2.35 / 24fps
      Music: ASTR - Operate
      (Et Aliae remix)
×
×
  • Create New...