Seja bem vind@, se você é um debiano (um baiano que usa debian) faça parte de nossa comunidade!
This is a follow-up to my previous post related the DebConf videoteam using a new software stack for the next conferences: http://acaia.ca/~tiago/posts/c3video-for-dc-take-4/.
This is about the encoding step from C3TT, mostly done by the script named D-encoding.
We can have many different encoding templates in the system. They're XSLT files which will generate the XML needed to create the local encoding commands. We can have more than one encoding template assigned for a conference.
XSLT encoding templates
A general comment: each meta ticket (say, the original ticket with meta info about the talk) will generate two children tickets over time, a recording one and a encoding one, with their own states. If things go wrong, a ingest ticket is created. More details can be checked here.
So I've got the proper encoded files in my Processing.Path.Output directory. The ticket is marked as encoded by the script. There's also a postencoding step, executed by E-postencoding. As I could understand, it's intended to be a general post-processing hook for encoded files. For instance, it can produces an audio file and make it available on Auphonic service. As we won't use that, we may want to set the property Processing.Auphonic.Enable to no.
The next step starts from the operator side. Just going to the Releasing tab in the web interface, choosing the ticket to check and doing a quick review in the encoded file:
Then, if everything looks fine, click Everthing's fine:
Check encoded file
From this point the ticket will be marked as checked and the next script (F-postprocessing) will take care of pushing the video(s)/audio(s) to the target place, as defined by Publishing.UploadTarget. I had to set myself the encoding template property EncodingProfile.Extension. We can optionally set Publishing.UploadOptions. (keep that in mind, seems not documented elsewhere)
So I have now the first DebCamp encoded video file uploaded to an external host, entirely processed using the C3TT software stack. We may also want to use a very last script to release the videos (eg. as torrents, to different mirrors and other onlive services) if needed. This is script-G-release.pl, which unlike others, won't be run by the screen UI in the sequence. It has some parameters hardcoded on it, although code is very clear and ready to be hacked. It'll also mark the ticket as Released.
That's all for now! In summary: I've been able to install and set C3TT up during a few days in DebCamp, and will be playing with it during DebConf. In case things go well we'll probably be using this system as our video processing environment for the next events.
We can access most CCC VoC software from https://github.com/voc. By having a look at what they're developing I feel that we (DebConf and CCC) share quite the same technical needs. And most important, the same spirit of community work for bringing part of the conference to those unable to attend.
DebCamp was warm!
This is a follow-up to my previous post related the DebConf videoteam using a new software stack for the next conferences: http://acaia.ca/~tiago/posts/c3video-for-dc-take-3/.
As mentioned before, C3TT provides a set of scripts which will work in background for most reviewing and video processing tasks. The first will just check if the talk is done and mark the related ticket as recorded.
The second script, B-mount4cut, does a nice job by mounting a custom fuse filesystem providing the following files (more detailed explanation here:
uncut.dv: full original dv file used as input file for the final trimming.
project.kdenlive: Kdenlive project file for the operator. Once it's saved with the trim marks, fuse-vdv will do a parse on it and use the marks for cutting.
cut.dv: contains only the frames between the trim marks extracted from project.kdenlive.
cut-complete.dv: contains the frames between the trim marks extracted from project.kdenlive, plus a prepended intro and an appended video. Paths for these files should be previously set in the web interface as properties Processing.Path.Outro and Processing.Path.Intros. The outro file can be, for instance, an appended video file with a common thanks for sponsors. The intro files is usually an individual frame for each talk, being a colorful presentation poster. We can also set the intro duration in the Processing.Duration.Intro property.
cut.wav: demuxed audio from cut.dv
Note: in fact, fuse-vdv provides virtual video files as a concatenation of original input files, so avoiding copying large and redundant data. Ideally, these fuse mountpoints will be shared over the network for operators via glusterFS, but I'll skip that for now.
After adding the trimming marks and saving the project using Kdenlive, the operator should go to the web interface and mark the ticket as cut:
Mark ticket as cut
Note: I've been able to do the marks in Kdenlive using double-click in the video, then editing the crop start/end options. The buttons "[" and "]" didn't work in my Kdenlive version for some unknown reason.
In the current DebConf video environment I had to make a link builder for translating the path/file names to the C3TT format. So, for the Amsterdam/2015-08-13/09:57:02.dv we should have a amsterdam-2015.08.13-09_57_02.dv symlink.
From now the system will delivery the next tasks to C-cut-postprocessor script. This script will just check the marks from the Kdenlive project file and do the cutting job. So far it has worked perfectly here for the first video sessions in DebCamp, with zero hack in the original code, wow!
Next post will be about the encoding script, named D-encoding.
This is a follow-up to my previous post related the DebConf videoteam using a new software stack for the next conferences: http://acaia.ca/~tiago/posts/c3video-for-dc-take-2/.
An outdated documentation for current subject is available at https://wiki.fem.tu-ilmenau.de/streaming/projekte/c3/28c3/crs/pipeline. Although the system may work differently nowadays, the basic idea remains the same. A newer, but incomplete documentation can be found in https://repository.fem.tu-ilmenau.de/trac/c3tt/wiki. Btw, CCC people from #voc at hackint.eu have been very kind and supportive.
I've set an instance of C3TT for DebConf15 in http://c3tt.acaia.ca/. If you want to play with it just ping me in #debconf-video at oftc. As you can see, we can keep a single external C3TT server for all Debian events, without much work left to the local side, doesn't it sound amazing?
Setting a new conference
Go to Projects, then Create.
In the project area we'll need to import the Tickets. Tickets will come from the schedule file, which is a XML as generated from frab. With a minor hack we've been able to make the schedule XML from DebConf Summit quite compatible to it (kudos cate!):
By importing the schedule from https://summit.debconf.org/debconf15.xml we'll be asked from which rooms we want to import the events. Usually those that have video coverage will be selected:
Then, we may want to exclude some talks that we won't provide video:
We're also required to adjust some Properties for a given conference. An example with some explanation of these properties is availabe at https://c3voc.de/wiki/c3tracker. For my initial tests the ones below seem to be enough:
The backend: basic understand
The screen UI mentioned above will run a set of scripts in background which will automate most of the tasks, preparing videos for cutting to deploying them in different online services.
Tab 0: A-recording-scheduler
Each 30 seconds it will check if there's any ticket in the state scheduled or recording. It's based in the start/end datetime of the talk, so the ticket will be kept as scheduled (current < talk start), or marked as recording (start => current =< end) or recorded (current > end).
Tab 1: B-mount4cut
Each 30 seconds it will check if there's any ticket in the state recorded. That means the talk is already finished, and the raw video file is available in the path which has previously been set as a property (Path.Capture) in the web interface.
For each ticket marked as recorded it will try to find the related video file in the capture path. The file format should be <room>-YYYY-MM-DD_HH-MM-SS.<extension>. The script will then use fuse-vdv to create a custom filesystem with some needed files for human interaction (fancy stuff!).
Here's an example of talks in a room called Heidelberg, after being recorded and auto-mounted by the B-mount4cut script:
Mounting custom fuse FS
The human interaction is just a short review process using Kdenlive. The reviewers will access these files via a glusterFS network share. There's even a Debian Virtualbox image provided for that, including all the necessary tools. I'm going into this right now and will report what I get in the next hours.
Hopefully the following scripts will also be covered, very soon-ish :)
Tab 2: C-cut-postprocessor
Tab 3: D-encoding
Tab 4: E-postencoding (auphonic)
Tab 5: F-postprocessing(upload)
DebCamp is fun.
This is a follow-up to my previous post related the DebConf videoteam using a new software stack for the next conferences: http://acaia.ca/~tiago/posts/c3video-for-dc-take-1/.
Installing C3TT scripts
There's a video (in German) which gives an idea about how the C3TT works: https://www.youtube.com/watch?v=K-KHbAcTo9I
It basically gives the volunteers a web interface to cut and review the recordings, wich communicates with a set of scripts running in background which will automate some tasks.
"Installing" the set of scripts is just a matter of placing them in a common directory and installing some Perl dependencies, mostly which are already packaged for Debian.
First check it out from the svn repository (fun fact: the web interface is coded in php in a git repository, the scripts are mostly written in perl with a little of bash, in a subversion repository. Both the conference and media system is are in ruby :)
$ svn co https://subversion.fem.tu-ilmenau.de/repository/cccongress $ mkdir /opt/crs; mv cccongress/trunk/tools /opt/crs/
A few libraries required:
$ apt-get install libboolean-perl libmath-round-perl libdatetime-perl libwww-curl-perl libconfig-inifiles-perl libxml-simple-perl $ perl -MCPAN -e 'install XML::RPC::Fast'
In the web tracker create a project, go to all projects => workers and create a worker (I'll try to explain it later). Go edit the worker and we'll see the token and secret that should be used in the scripts to talk to the interface.
Create a file tracker-profile.sh with the follow lines (using our correct values):
export CRS_TRACKER=http://localhost/rpc export CRS_TOKEN=2q24M7LW4Rk31YNW4tWKv8koNvyM3V4s export CRS_SECRET=5j8SyCS35W2SBk2XIM4IWeDUqF9agG1x
We also need to build and install the fuse-vdv package from trunk/tools (if working with dv files, otherwise fuse-ts package).
Next step is run the scripts. Fortunatelly a nice UI has been done using screen with multiple tabs, which can be alternated using a <Ctrl+a> <number> combination.
cd /opt/crs/tools/tracker3.0 && /start-screenrc-dv.sh
We'll get the following:
Screen tabs from C3TT
In a next post I'll try to explain a bit of how the web system work together with the scripts and how to do a basic setup for a real conference. I hope to get there soon!
DebConf has provided live streaming and recordings from talks since 2004. We used to work with a set of scripts which worked together with Pentabarf for most of videoteam tasks, including volunteer shifts coordination, reviewing process, encoding and deployment.
Things has changed since DebConf14, when Pentabarf was replaced by Summit as the conference management system. Without those old Pentabarf features and hacks we had to invent new ways of dealing with the video workflow in DebConf. We gave veyepar a try in 2014, and we will probably do it again in DebConf15. However, for a long term solution we are considering the software stack from CCC Video Operation Center, which so far I see as a free, solid and community-oriented mix of Debian-friendly tools.
I will be reporting the progress on setting up and testing the CCC software strucutre for DebConf. Having the opportunity of being in DebCamp together with other videoteam folks will certainly make things easier :)
Setting up the CCC Ticket Tracker
C3TT is a ticket/tracker system used by CCC for reviewing/encoding process.
The web side of C3TT is written in PHP and can be cloned from http://git.fem.tu-ilmenau.de/cccongress.git. Some documentation is available at https://repository.fem.tu-ilmenau.de/trac/c3tt/wiki and from https://c3voc.de/wiki/c3tracker.
What I've done so far to get it working:
Installing some dependencies:
$ apt-get install postgresql-9.4 php5-pgsql php5-xsl postgresql-contrib-9.4 php5-xmlrpc php5
Creating database and users:
$ su -s /bin/bash postgres $ createuser -DRS dc15 $ createdb -O dc15 c3tt $ psql postgres=# ALTER ROLE dc15 WITH PASSWORD 'xxx';
Basic site config using lighttpd:
Running the installer script:
$ php -q Install/install.php
This will ask you some questions, then will create the config file and populate the database. At this point you should be able to access the ticket track system from your browser.
The set of scripts from C3TT doesn't need to be installed in the same host as the web side, they communicate via XML/RPC. In a next post I will report the installation and initial setup for these scripts.