dcinemaforum.com

Production => Digital Workflow => Topic started by: Martini77 on May 14, 2012, 08:37:04 PM

Title: Getting the DCP onto an appropriate drive...
Post by: Martini77 on May 14, 2012, 08:37:04 PM
Hi there - I'm new to DCP and have just created my first using OPENDCP on a feature.

It was surprisingly easy (!) though I have yet to test it; that's happening at a movie theatre later this week. Or at least it's supposed to be happening....

I was aiming to get the DCP onto an EX2 drive but so far I've not had much luck.

I'm using a Mac OSX 10.7.4

I've rooted around the internet but can't find a solution to formatting an  external USB drive as EX2. (I have installed FUSE For OSX and fuse-ex2)

Does anyone have any solutions or leads? Preferably not involving code in any form other than the most simple kind - I'm a filmmaker and already on the edge of my computer tech know-how getting as far as the DCP.

Thanks for reading and in advance of any help you can offer.
Martin
Title: Re: Getting the DCP onto an appropriate drive...
Post by: walterlionguy on May 15, 2012, 03:06:32 AM
Hey Martin,

If you are having trouble getting the Ext2 File system put onto an external drive, I say the easiest and most fool proof way to do it is to use a Linux Live DVD.  Ubuntu Desktop is simple and is basically fool proof for doing just this.  You can get the latest ISO from their website (http://www.ubuntu.com/download).  Burn the image to a disc, and boot from it.  From there you can format the drive to Ext2 and copy your DCP from your local disk to the external drive while in the Ubuntu Live session.  When your done just reboot and pop out the disc, simple and no need to install anything.

Any questions, feel free to ask! :)

Regards,
Walter
Title: Re: Getting the DCP onto an appropriate drive...
Post by: Martini77 on May 15, 2012, 01:44:36 PM
Hi Walter - thanks for your prompt reply - very helpful.

I will try this - but thinking ahead - will I be able to mount the mac osx drives whilst booted in Ubuntu?

thanks again
Martin
Title: Re: Getting the DCP onto an appropriate drive...
Post by: walterlionguy on May 15, 2012, 02:37:05 PM
Hi Martin,

You should be able to mount and read from an OSX HFS formatted drive within linux, not sure about write though.  I know it reads NTFS drives too.  Let us know how it turns out.

Walter
Title: Re: Getting the DCP onto an appropriate drive...
Post by: Martini77 on May 15, 2012, 03:43:18 PM
Thanks again, Walter.

I've booted in Ubuntu. I can see my MacOSX drives and I'm currently copying across the DCP. I feel good.

Do you happen to know, can I place multiple DCP folders on the same Ext2 drive?
(Of course I'd follow naming conventions to distinguish them).
My film is in English but the accents area very strong, so ideally I'd have a 2nd DCP on the drive that has English subtitles, giving international exhibitors the option.

BTW - I promise I won't keep adding on subsequent questions...
thanks again.
Martin
Title: Re: Getting the DCP onto an appropriate drive...
Post by: walterlionguy on May 15, 2012, 04:13:38 PM
Don't worry, ask all the questions you need!  Many users here including myself have fun helping others out!

As for the multiple DCP folders on one Ext2 drive , I have tried this several times before on Doremi/Dolby servers using both Interop and SMPTE without any problems.  It worked fine on Doremi TMS too.  Just make sure the DCP folders are placed onto the drive and not into sub folders.
Title: Re: Getting the DCP onto an appropriate drive...
Post by: Martini77 on May 17, 2012, 05:17:15 PM
I've looked at the DCP in a 250seater theatre today and it looked great - so that's a relief.
Thanks for your guidance, Walter.
Best wishes
Martin
Title: Re: Getting the DCP onto an appropriate drive...
Post by: happytomy on September 12, 2012, 12:17:04 AM
Thanks again, Walter.

I've booted in Ubuntu. I can see my MacOSX drives and I'm currently copying across the DCP. I feel good.

Do you happen to know, can I place multiple DCP folders on the same Ext2 drive?
(Of course I'd follow naming conventions to distinguish them).
My film is in English but the accents area very strong, so ideally I'd have a 2nd DCP on the drive that has English subtitles, giving international exhibitors the option.

BTW - I promise I won't keep adding on subsequent questions...
thanks again.
Martin
HI Martini,I use Macbook Pro also,Would you like to give all the information about using the ubuntu live cd to boot the mbp and how to copy the dcp folder to the ext2 removable disk,thx.^-^
Title: Re: Getting the DCP onto an appropriate drive...
Post by: hdvprojection on September 12, 2012, 08:36:54 AM
NTFS is your best bet. The individual screen servers are almost always Linux, so ext2/3/4 are good there, however the LMS/TMS are very often Windows-based. They won't mount Linux or OSX file systems.
Title: Re: Getting the DCP onto an appropriate drive...
Post by: walterlionguy on September 12, 2012, 03:05:13 PM
Quote
NTFS is your best bet. The individual screen servers are almost always Linux, so ext2/3/4 are good there, however the LMS/TMS are very often Windows-based. They won't mount Linux or OSX file systems.

No, just use a Linux live DVD or make a bootable live USB and stick to Linux ext2/3/etc (assuming you don't have Linux already).  This is the only way to absolutely guarantee 100% that all DCI servers will read your content.  I have never even seen a windows based LMS setup before, and even if the manufacturer decided to use windows they would be forced to support the Linux ext filesystem as its the main standard for this.  All the major distributors (such as Technicolor) use Linux ext formatted drives and I have never even seen an NTFS drive from them.  If they were using NTFS drives its going to eventually hit some users who find that it isn't working for them.  Of course if the whole entire industry demanded a change to NTFS then that would be a different story... but this is not the case.

There is nothing wrong with using NTFS if it works for you, but there is no point in posting information telling everyone that it's their best bet, when its obviously not.  Besides that its almost trivial to make a Linux live DVD/USB anyways...
Title: Re: Getting the DCP onto an appropriate drive...
Post by: hdvprojection on September 13, 2012, 12:30:29 AM
The LMS console that Christie puts together for GDC (and almost certainly for other brands of servers) is Windows-based, and won't accept DCPs on ext3 hard drives. You have to go to the individual houses' servers to ingest ext3. NTFS (or FAT32, I suppose) at the LMS.

I'm not making this up. I have no reason to lie or spread false information. Just sharing what I've learned and experienced.
Title: Re: Getting the DCP onto an appropriate drive...
Post by: loki on October 10, 2012, 11:06:55 AM
I am also about to distribute some copies of DCPs from ext2 and ext3 drives and for this reason I am creating some test drives right now.

Because we're just starting creating and copying our own DCPs I didn't order CRUs yet and use simple ext2/3 formatted hard drives without cases. The injest process should still work if the TMS uses Linux, right?

Another technical detail I read about is the inode size. Many people write the compatibility with ext2/3 drives that use inode size 128 is higher than drives formatted with 256 (due to other programs involved on Win/Mac systems). So now I am using openSUSE for formatting drives into ext2/3 and the standard inode size got increased to 256 in 2008s openSUSE 11.0 release. Do you know if the incompatibility is still a current topic? Could it produce errors if I distribute DCPs on ext2/3 hard drives with inode size 256?
As far as I know there is also no way changing the inode size without formatting?

I'll let you know what happened after the test anyway.
Title: Re: Getting the DCP onto an appropriate drive...
Post by: drmrr on October 11, 2012, 03:15:20 AM
Do you know if the incompatibility is still a current topic? Could it produce errors if I distribute DCPs on ext2/3 hard drives with inode size 256?
QUBE XP-D (at least version 2.5.5.2) will not read HDD at all if inode size is larger than 128b unless you update its dodgy EXT3 filesystem driver manually (the server itself is Windows XP so this is not a problem if you know the admin password or can "hack" it using flash boot).
Title: Re: Getting the DCP onto an appropriate drive...
Post by: drmrr on October 11, 2012, 03:28:01 AM
Because we're just starting creating and copying our own DCPs I didn't order CRUs yet and use simple ext2/3 formatted hard drives without cases. The injest process should still work if the TMS uses Linux, right?
Right. CRUs are way overrated, overpriced and overweight. You need them only if you are going to write tens of HDDs simultaneously in HDD duplicator. Otherwise USB3 portable disk is cheaper and more robust (and survives more stress in transporting; I've seen 3.5'' HDD broken in delivery even in protective cases but none of portable 2.5'' ones).

Quote
Could it produce errors if I distribute DCPs on ext2/3 hard drives with inode size 256?

Code: [Select]
mke2fs -j -I 128 /dev/sdb9will make you more confident (this command will reformat the drive, you can't change inode size other way).
Title: Re: Getting the DCP onto an appropriate drive...
Post by: loki on October 11, 2012, 06:10:28 AM
drmrr, thank you for sharing your experience.

about the inode you wrote:

Code: [Select]
mke2fs -j -I 128 /dev/sdb9will make you more confident (this command will reformat the drive, you can't change inode size other way).

In openSUSEs Partitioner there is an advanced preference with which you are able to change the inode size, too. The only problem I encountered yesterday was that only "root" had permissions to write on the new 128-formatted drive. In the disc utility I can choose to become the new owner of the drive while formatting. Do you have an idea on how to change from "root" to "userxy" on openSUSE? I already tried "chown" but it said I had no permission to change permissions. If there is a way I can get that DCP on the drive anyhow, I wouldn't change the permissions.. Just want to be able to copy it to the drive and let the booth manager play it.
Title: Re: Getting the DCP onto an appropriate drive...
Post by: drmrr on October 11, 2012, 11:23:30 AM
Do you have an idea on how to change from "root" to "userxy" on openSUSE? I already tried "chown" but it said I had no permission to change permissions.

Code: [Select]
sudo chown -R userxy:userxy mounted_disk_root_path
should help.
Title: Re: Getting the DCP onto an appropriate drive...
Post by: freedcp on October 12, 2012, 05:13:19 AM
If there is a way I can get that DCP on the drive anyhow, I wouldn't change the permissions.. Just want to be able to copy it to the drive and let the booth manager play it.

Hi,

I think you go in a wrong way by changing permissions on the hard drive.
DCPs disks should always be owned by root by default as the DCP files on it (and 775 for permissions).
It's designed for protect files to not be altered by anyway if people try to mount the hard drive on a computer (.hidden files for example who generate issues on SmartJog it seems).
The only think to do is making all operations on the disk as root (with sudo).

Now for the copy, i suggest to copy files from CLI with rsync + checksum verification, not from your desktop interface (gnome, kde, lxde, etc).
If you go this way, you just have to make the operation as root (with sudo) and you don't need to mess with the hard drive ownership/permission.
This is how i copy all my DCPs on hard drives:
Code: [Select]
sudo rsync -rvhc --progress MY_DCP_FOLDER /media/MY_HARD_DRIVE/Note the DCP folder don't finish with a slash when the hard drive folder finish with one. It's important or you will copy all the files directly into the hard drive and not into a folder on the hard drive !
After, just verify (with ls -l) than the DCP folder is 775 or chmod it (sudo chmod -R 775 /media/YOUR_HARD_DRIVE/YOUR_DCP_FOLDER)

Until now I never got any issues with the DCPs i copied this way.
Title: Re: Getting the DCP onto an appropriate drive...
Post by: loki on October 12, 2012, 08:17:28 AM
Hi,

thank you for the information. I'll try this process next week.

Also thank you for offering such a nice website (freedcp.net). It's pretty useful.
Title: Re: Getting the DCP onto an appropriate drive...
Post by: hdvprojection on October 12, 2012, 04:01:30 PM
Anyone know if ext4 works for any of the commercial servers out there?
Title: Re: Getting the DCP onto an appropriate drive...
Post by: freedcp on October 21, 2012, 05:09:12 AM
Anyone know if ext4 works for any of the commercial servers out there?

I don't know as i never tried, but the DCI spec. says it should be ext2/ext3:
Quote
The distribution media partition shall be formatted in either the EXT2 or the EXT3 format. When the file system is formatted, the inode size shall be set to 128 bytes.
http://www.isdcf.com/ISDCF/DiscFormat.html (http://www.isdcf.com/ISDCF/DiscFormat.html)


Now as a personnal point of view, i would say avoid ext4.
It's a "new" file system and you can't be sure every servers will handle it correctly, specially the old one or the one running on Windows (GDC?) who have to use a "driver" (like ext3fsd) for reading linux drives.
Title: Re: Getting the DCP onto an appropriate drive...
Post by: Polona on November 17, 2012, 04:43:31 PM
The LMS console that Christie puts together for GDC (and almost certainly for other brands of servers) is Windows-based, and won't accept DCPs on ext3 hard drives. You have to go to the individual houses' servers to ingest ext3. NTFS (or FAT32, I suppose) at the LMS.

I'm not making this up. I have no reason to lie or spread false information. Just sharing what I've learned and experienced.

NTFS is not DCI. The hard drive must be formatted ext2 or ext3 according to the DCI. If the LMS is not accepting DCI formatted ext2/ext3 disks, you should contact the provider and the device might need an update.

I work a lot for film festival and we are experiencing big problems with disks formatted in a wrong way with wrong permissions.
When it's not DCI, it might work at the local theatre but it might not work somewhere else, so make sure it is according to DCI.

Right. CRUs are way overrated, overpriced and overweight. You need them only if you are going to write tens of HDDs simultaneously in HDD duplicator. Otherwise USB3 portable disk is cheaper and more robust (and survives more stress in transporting; I've seen 3.5'' HDD broken in delivery even in protective cases but none of portable 2.5'' ones).

Small usb disks are not better than cru. At a festival when you have to load 200 DCPs it costs ages to copy via USB.
Please, if you have a chance and budget, use a cru for sending to festivals or at least buy a faster disk that also has esata connection.
Taking care the DCPs are not waaay to big is also very appreciated. I saw a feature 2D 2K DCP that was 250GB, more than twice as much as an average feature film. It took hours to load it!
Title: Re: Getting the DCP onto an appropriate drive...
Post by: hdvprojection on November 17, 2012, 05:01:44 PM
NTFS is not DCI. The hard drive must be formatted ext2 or ext3 according to the DCI. If the LMS is not accepting DCI formatted ext2/ext3 disks, you should contact the provider and the device might need an update.

Small usb disks are not better than cru. At a festival when you have to load 200 DCPs it costs ages to copy via USB.
Please, if you have a chance and budget, use a cru for sending to festivals or at least buy a faster disk that also has esata connection.
Taking care the DCPs are not waaay to big is also very appreciated. I saw a feature 2D 2K DCP that was 250GB, more than twice as much as an average feature film. It took hours to load it!

I've been formatting all of my thumb drives and large portables in NTFS and have yet to have a problem anywhere. Look, it's simple: Linux systems are smart enough and flexible enough to deal with NTFS. Windows... not so much. You have to prepare for idiocy like Microsoft's and others. Sometimes if the thumb drive is large enough I will have dual partitions with dupes on both. The NTFS partition has never failed.
Title: Re: Getting the DCP onto an appropriate drive...
Post by: walterlionguy on November 17, 2012, 06:48:35 PM
Polona is actually right.  You really don't want to be always using a file system that is not officially supported by DCI.  Sure its true that the majority of DCI servers today will read NTFS but you could still run into a server running an old FW version that may not read NTFS.  Besides that, NTFS itself as a filesystem is kinda lousy if you ask me.  It has no journaling and therefore is prone to fragmentation, especially when the drive gets filled in the ~80%+ range.  At my theatre every single film I have ever seen by a reputable distributor has used ext2/ext3 formatted drives.

The only times I have honestly seen NTFS commercially used for DCP's are when we receive low budget films, which are usually just on small external USB3 drives.

As for the external USB3 drives, I try to avoid them for speed sake right now.  It's by no means fast waiting for 200+GB DCP's to load over USB2.0 and unfortunately that seems to be the case as USB3 isn't really supported in the DCI world yet.  I don't imagine that DCI will ad USB3 in their specification, but will just wait for the motherboard manufacturers to just start adding it themselves more as time moves on.  Similar to how USB2.0 slowly replaced USB1.1 back in the day.
Title: Re: Getting the DCP onto an appropriate drive...
Post by: Terrence Meiczinger on November 17, 2012, 07:47:52 PM
The DCI does not specify any drive format... or transport mechanism for that matter. NTFS is as DCI compliant as EXT3, however in practice, EXT3 is most common primarily due to the use of linux based servers.
Title: Re: Getting the DCP onto an appropriate drive...
Post by: drmrr on November 19, 2012, 08:14:25 AM
NTFS itself as a filesystem is kinda lousy if you ask me.  It has no journaling and therefore is prone to fragmentation

You are wrong in all possible ways. NTFS has pretty robust journaling in fact. And fragmentation is not a thing that is related to journaling in any way.

Also, the majority of drives "in the wild" from big distributors and studios are EXT2, not EXT3. No journaling in sight at all.

All current cinema servers with more-or-less current firmware read EXT and NTFS without a hitch. And singe it is as easy to access EXT drive under Windows as NTFS drive under Linux I think this is a non-issue now.

Also all USB3 frives are as perfectly accessible for cinema servers as their USB2 counterparts since USB2 signal compatibility is required and maintained for every USB3 device in the world.
Title: Re: Getting the DCP onto an appropriate drive...
Post by: hdvprojection on November 20, 2012, 01:18:14 AM
And singe it is as easy to access EXT drive under Windows as NTFS drive under Linux I think this is a non-issue now.
I think it very soon will be. However, there are still some old servers out there being rented out for screenings, festivals and special events that have not been upgraded regularly, perhaps never were and never will be. So it's still somewhat of an issue, probably for the next year or two at least.

And by the way, I'm on a brand new Windows 7 64-bit system at work with all of the bells and whistles installed on it, and it's not capable of mounting an ext3 thumb drive, at least not yet. Linux can mount almost anything OotB. Windows cannot. That's really my only point.
Title: Re: Getting the DCP onto an appropriate drive...
Post by: walterlionguy on November 20, 2012, 05:29:05 AM
Quote
Also all USB3 frives are as perfectly accessible for cinema servers as their USB2 counterparts since USB2 signal compatibility is required and maintained for every USB3 device in the world.

When did anyone here mention a capability problem?  Of course it's backwards compatible with v2/1.1.  It's just really slow over USB2 when you have big files to copy.
Title: Re: Getting the DCP onto an appropriate drive...
Post by: somewhoo on April 01, 2013, 05:35:18 PM
Because most commercial servers are linux based machines as already someone mentioned, best way is to maintain three things:
-partition table must be MBR
-inode size should be 128, and
-drive should be ext3 file system
I've got many DCP packed and I try to do it only like this and I didn't had any problems or complains with it.. So I do it in linux (open suse) following these steps:
-plug new disk (usb2/3 (portable ones) or e-sata (CRU))
-open GPrated (install it from online repo)
-check witch disk drive is new one, in my case it's sdd (I've got two internal disks, sda and sdb with one partitions per drive (sda1 and sdb1)
-deleted default partition (deleted sdd1 wich is default partition witch is external portable drive formatted by default), apply changes in GParted
-make default partition table for empty drive and make it MBR
-make new partition and set format to ext3 (leave all settings default)
-apply changes in GParted and when format is complete close it
-open terminal as root (type sudo su and again type your pass and hit enter)
-format that partition again but this time set inode size to 128 (type mkfs.ext2 -j -I 128 /dev/sdd1 and hit enter)
-now mount your partition with sudo mount /dev/sdd1 /mnt/disk and make directory for your dcp with sudo mkdir /mnt/disk/dcp
-make permissions for that directory so that you can write to it with sudo chmod -R 777 /mnt/disk/dcp
-copy dcp files (assets) to that directory
-and when copying finishes make that directory read only and executable with this command chmod -R 755 /mnt/disk/dcp
-and last step unmount disk with sudo umount /mnt/disk

Ofcourse make changes in commands with appropriate disk device name (in my case is "sdd1" for partition and "sdd" for disk drive..
And if you can't or wont install linux box you can use GParted live cd from
Code: [Select]
http://gparted.sourceforge.net/livecd.phpMore info
Code: [Select]
http://www.isdcf.com/ISDCF/DiscFormat.htmlcredits, lost source :(

Best :)
Title: Re: Getting the DCP onto an appropriate drive...
Post by: edzus on July 31, 2015, 05:29:04 AM
normally the format is ext3. If you dont have linux to preformat drive or just dont want to do it yourself, you can find dcp different submition standarts here: http://www.span.com/feature/DCP__kits_line___CRU__data__port__cinema__pack~1293
Title: Re: Getting the DCP onto an appropriate drive...
Post by: edzus on July 31, 2015, 05:31:28 AM
normally the format is ext3. If you dont have linux to preformat drive or just dont want to do it yourself, you can find dcp different submition standarts here: http://www.span.com/feature/DCP__kits_line___CRU__data__port__cinema__pack~1293
Title: Re: Getting the DCP onto an appropriate drive...
Post by: hdvprojection on February 04, 2017, 03:58:36 AM
I'm re-reading this thread and some of my contributions, which I feel are unclear. So...

Many, if not most, or even close to all TMS/LMS systems at theaters are Windows-based. Many of these do not have the capability to read Ext2/3 out-of-the-box (I think Hollywood Software's TCC3 does, and probably Arts Alliance). This is the reason that the labs (Fotokem, Deluxe, Technicolor, etc.) send movie trailers out on thumb drives formatted NTFS, not Ext2/3. Think about this: they get the same 4GB or 8GB drives we do from the store or wherever, which are FAT32 when you buy them. They go to all of the trouble of re-formatting these drives, but NOT to Ext2/3. They go with NTFS... Why?

This has nothing to do with DCI specs, but rather with the theater/library management software. I'd be shocked to find out if there are any DCI requirements for TMS systems; they just have to be on the media and/or management network and they can do what they like. At most multiplexes the TMS/LMS is the main point of contact with the servers. Whatever you're producing will most likely be ingested from there. Make your drives NTFS-friendly, especially if we're talking about ads, trailers, shorts, etc.

With features I think it's a more fair point to make that Ext3 is the way to go. But then, the theater staff deal with features differently, too. Trailers they want to ingest and schedule at one sitting. Features, meanwhile, they plug in to ingest and then walk away for a few hours, going about their business.