Please login or register. January 24, 2019, 01:03:27 AM

Author Topic: OpenDCP performance / issue  (Read 4124 times)

eriklodr

  • New Member
  • **
  • Posts: 10
    • View Profile
OpenDCP performance / issue
« on: February 09, 2016, 05:54:46 AM »
Hello!

I've a problem, atleast I think so.
I'm using OpenDCP to convert 110,00 frames (a feature film), to j2c. XYZ color transformation has already been done.
" opendcp_j2k -i xyztiff -o j2c -r 24 -p cinema2k -b 125 -x -t 12 "

This is using a XEON 5820K (6/12 core), I find it, that it takes a very long time, 1 day and 15hours, don't know if that's normal. Thing is, that OpenDCP dosn't even tax my CPU on this. I have/had a Sandy Bridge i7-2600K, which I've used before doing some short films, on that machine OpenDCP was taxing the shit out of it, 100% CPU load, and I found it fast, if memory serves me right.

And, I'm using two (2) WD Re's to read and write to each other, so I get good seqential performance, maybe that's overkill.

Question is.. is this normal?
Is there some settings I can tweak? Is my system "corrupt"?
Yes, I've used the "OMP_NUM_THREAD" setting.

Running, Windows 7 64it, 32GB Ram, 5820K (OC'd to 4GHz)

Terrence Meiczinger

  • Administrator
  • Hero Member
  • *****
  • Posts: 560
    • View Profile
Re: OpenDCP performance / issue
« Reply #1 on: February 09, 2016, 05:14:20 PM »
110,000 frames / 140,400 seconds = .78 fps which is REALLY REALLY slow for a system like that. I can get around 4 fps on my 6 year old iMac i7 quad-core w/8GB using the the internal drive. Can you see the per CPU/Core usage? I don't know Windows, so I don't if that is possible. OpenDCP may only be using 1 thread. If you add --log-level 4 to the opendcp_j2k command, it will give you information including the number of threads it spun up.

Have you tried the GUI version?

eriklodr

  • New Member
  • **
  • Posts: 10
    • View Profile
Re: OpenDCP performance / issue
« Reply #2 on: February 10, 2016, 08:54:56 AM »
Thanks for the reply Terrance.

I ran with "--log-level 4" it says it using 12 Threads "JPEG2000 Conversion (12 threads)". But I don't belive it, it's slow!
Ran the GUI, CPU is pumping at 4GHz!!! :) 90-100% load. Did 1000 frames in a flash. I'd like to get to the bottom of this, not sure how. I have  tried the env variables "OMP_NUM_THREADS" thing, still the same, like I'd said before. Thing is, I'd like to use the command prompt for automation, I'm using graphicsmagick before, to do some scaling. Come to think of it, I wonder if that is slow also, hmmm.

Edit:
I also tested the command line interface under linux/Ubuntu using a terminal window under X (did not try the GUI, was in hurry), very slow also. Ubuntu 15.10.

110,000 frames / 140,400 seconds = .78 fps which is REALLY REALLY slow for a system like that. I can get around 4 fps on my 6 year old iMac i7 quad-core w/8GB using the the internal drive. Can you see the per CPU/Core usage? I don't know Windows, so I don't if that is possible. OpenDCP may only be using 1 thread. If you add --log-level 4 to the opendcp_j2k command, it will give you information including the number of threads it spun up.

Have you tried the GUI version?
« Last Edit: February 10, 2016, 08:57:20 AM by eriklodr »

Terrence Meiczinger

  • Administrator
  • Hero Member
  • *****
  • Posts: 560
    • View Profile
Re: OpenDCP performance / issue
« Reply #3 on: February 10, 2016, 12:51:21 PM »
Ok, is there a way to see how many cores/threads an application is using under Windows? OpenDCP is trying to use 12 threads, but that doesn't mean Windows is letting it. The GUI and CLI use different mechanisms for multi-threading, so its possible the CLI has issues on certain systems.

eriklodr

  • New Member
  • **
  • Posts: 10
    • View Profile
Re: OpenDCP performance / issue
« Reply #4 on: February 10, 2016, 06:16:56 PM »
You can always start the TaskManager :) I'll look into that, must be some kind of debuger out there. I'll get back on this on Friday.

I'll recheck the CLI (vs GUI) on Linux also, I found it slow as well.

Thanks so far.

Ok, is there a way to see how many cores/threads an application is using under Windows? OpenDCP is trying to use 12 threads, but that doesn't mean Windows is letting it. The GUI and CLI use different mechanisms for multi-threading, so its possible the CLI has issues on certain systems.

eriklodr

  • New Member
  • **
  • Posts: 10
    • View Profile
Re: OpenDCP performance / issue
« Reply #5 on: February 23, 2016, 04:40:38 PM »
Okey, sorry for the delay. Went on vaccation and had to do some other stuff.

Anways, I found out there was something wrong with the disk under Windows, the other WD RE drive performed well. I did a test on that other drive, was pretty good, I really don't know how Windows calculate threads information but it said it had "49" threads running using the "Task Manager". Didn't look to much into it. But it (opendcp_j2k) was certenlty doing it's job pretty well.

I booted into Ubuntu formated my drives to ext4 and mounted them. Everything seems to be running smooth.
Did a test with a one minute clip I extract from a movie. Scaled to 1998x1080 with XYZ conversion done before OpenDCP. 1440 frames, 24fps.

All threads are spining up, monitoring via "htop". it took about 4minutes-something, that's an 6 fps / per second. This was with the 5820k back to "normal" settings, not OC'ed. Turbo at 3.6GHz max. I can't stop to think, or feel, that that's slow(?).

dcinemaforum.com

Re: OpenDCP performance / issue
« Reply #5 on: February 23, 2016, 04:40:38 PM »