c3video for debconf #4

This is a follow-up to my previous post related the DebConf videoteam using a new software stack for the next conferences: http://acaia.ca/~tiago/posts/c3video-for-dc-take-3/.

As mentioned before, C3TT provides a set of scripts which will work in background for most reviewing and video processing tasks. The first will just check if the talk is done and mark the related ticket as recorded.

The second script, B-mount4cut, does a nice job by mounting a custom fuse filesystem providing the following files (more detailed explanation here:

uncut.dv: full original dv file used as input file for the final trimming.

project.kdenlive: Kdenlive project file for the operator. Once it's saved with the trim marks, fuse-vdv will do a parse on it and use the marks for cutting.

cut.dv: contains only the frames between the trim marks extracted from project.kdenlive.

cut-complete.dv: contains the frames between the trim marks extracted from project.kdenlive, plus a prepended intro and an appended video. Paths for these files should be previously set in the web interface as properties Processing.Path.Outro and Processing.Path.Intros. The outro file can be, for instance, an appended video file with a common thanks for sponsors. The intro files is usually an individual frame for each talk, being a colorful presentation poster. We can also set the intro duration in the Processing.Duration.Intro property.

cut.wav: demuxed audio from cut.dv

Note: in fact, fuse-vdv provides virtual video files as a concatenation of original input files, so avoiding copying large and redundant data. Ideally, these fuse mountpoints will be shared over the network for operators via glusterFS, but I'll skip that for now.

After adding the trimming marks and saving the project using Kdenlive, the operator should go to the web interface and mark the ticket as cut:

Mark ticket as cut

Note: I've been able to do the marks in Kdenlive using double-click in the video, then editing the crop start/end options. The buttons "[" and "]" didn't work in my Kdenlive version for some unknown reason.

In the current DebConf video environment I had to make a link builder for translating the path/file names to the C3TT format. So, for the Amsterdam/2015-08-13/09:57:02.dv we should have a amsterdam-2015.08.13-09_57_02.dv symlink.

From now the system will delivery the next tasks to C-cut-postprocessor script. This script will just check the marks from the Kdenlive project file and do the cutting job. So far it has worked perfectly here for the first video sessions in DebCamp, with zero hack in the original code, wow!

Next post will be about the encoding script, named D-encoding.

c3video for debconf #2

This is a follow-up to my previous post related the DebConf videoteam using a new software stack for the next conferences: http://acaia.ca/~tiago/posts/c3video-for-dc-take-1/.

Installing C3TT scripts

There's a video (in German) which gives an idea about how the C3TT works: https://www.youtube.com/watch?v=K-KHbAcTo9I

It basically gives the volunteers a web interface to cut and review the recordings, wich communicates with a set of scripts running in background which will automate some tasks.

"Installing" the set of scripts is just a matter of placing them in a common directory and installing some Perl dependencies, mostly which are already packaged for Debian.

First check it out from the svn repository (fun fact: the web interface is coded in php in a git repository, the scripts are mostly written in perl with a little of bash, in a subversion repository. Both the conference and media system is are in ruby :)

$ svn co https://subversion.fem.tu-ilmenau.de/repository/cccongress
$ mkdir /opt/crs; mv cccongress/trunk/tools /opt/crs/

A few libraries required:

$ apt-get install libboolean-perl libmath-round-perl libdatetime-perl libwww-curl-perl libconfig-inifiles-perl libxml-simple-perl
$ perl -MCPAN -e 'install XML::RPC::Fast'

In the web tracker create a project, go to all projects => workers and create a worker (I'll try to explain it later). Go edit the worker and we'll see the token and secret that should be used in the scripts to talk to the interface.

cd  /opt/crs/tools/tracker3.0

Create a file tracker-profile.sh with the follow lines (using our correct values):

export CRS_TRACKER=http://localhost/rpc
export CRS_TOKEN=2q24M7LW4Rk31YNW4tWKv8koNvyM3V4s
export CRS_SECRET=5j8SyCS35W2SBk2XIM4IWeDUqF9agG1x

We also need to build and install the fuse-vdv package from trunk/tools (if working with dv files, otherwise fuse-ts package).

Next step is run the scripts. Fortunatelly a nice UI has been done using screen with multiple tabs, which can be alternated using a <Ctrl+a> <number> combination.

cd  /opt/crs/tools/tracker3.0 && /start-screenrc-dv.sh

We'll get the following:

Screen tabs from C3TT

In a next post I'll try to explain a bit of how the web system work together with the scripts and how to do a basic setup for a real conference. I hope to get there soon!

Elixir in Debian, MiniDebconf at FISL, and Debian CI updates

In June I started keeping track of my Debian activities, and this is my July update.

Elixir in Debian

Elixir is a functional language built on top of the Erlang virtual machine. If features imutable data structures, interesting concurrency primitives, and everything else that Erlang does, but with a syntax inspired by Ruby what makes it much more aproachable in my opinion.

Those interested in Elixir for Debian are encouraged to hang around in #debian-elixir on the OFTC IRC servers. There are still a lot of things to figure out, for example how packaging Elixir libraries and applications is going to work.

MiniDebconf at FISL, and beyond

I helped organize a MiniDebconf at this year’s FISL, in Porto Alegre on the 10th of July. The whole program was targetted at getting more people to participate in Debian, so there were talks about translation, packaging, and a few other more specific topics.

I myself gave two talks: one about Debian basics, “What is Debian, and how it works”, and second one on “packaging the free software web”, which I will also give at Debconf15 later this month.

The recordings are available (all talks in Portuguese) at the Debian video archive thanks to Holger Levsen.

We are also organizing a new MiniDebconf in October as part of the Latinoware schedule.

Ruby

We are in the middle of a transition to switch to Ruby 2.2 as default in Debian unstable, and we are almost there. The Ruby transition is now on hold while GCC 5 one is going on, but will be picked up as soon as were are done with GCC 5.

ruby-defaults has been uploaded to experimental for those that want to try having Ruby 2.2 as default before that change hits unstable. I myself have been using Ruby 2.2 as default for several weeks without any problem so far, including using vagrant on a daily basis and doing all my development on sid with it.

I started taking notes about Ruby interpreter transitions work to make sure that knowledge is registered.

I have uploaded minor security updates of both ruby2.1 and ruby2.2 to unstable. They both reached testing earlier today.

I have also fixed another bug in redmine, which I hope to get into stable as well as soon as possible.

gem2deb has seen several improvements through versions 0.19, 0.20, 0.20.1 and 0.20.2.

I have updated a few packages:

• ruby-rubymail
• ruby-ferret
• ruby-omniauth
• ruby-hashie
• ruby-rack-accept
• chef-zero
• nailgun-agent
• ruby-serialport
• ruby-gnome2
• ruby-mysql2
• ruby-dataobjects-postgres
• ruby-standalone
• thin
• ruby-stringex
• ruby-i18n

Two NEW packages, ruby-rack-contrib and ruby-grape-logging ,were ACCEPTED into the Debian archive. Kudos to the ftp-master team who are doing an awesome job reviewing new packages really fast.

Debian Continuous Integration

This month I have made good progress with the changes needed to make debci work as a distributed system with one master/scheduler node and as many worker nodes (running tests) as possible.

While doing my tests, I have submitted a patch to lxc and updated autodep8 in unstable. At some point I plan to upload both autodep8 and autopkgtest to jessie-backports.

Sponsoring

I have sponsored a few packages:

• ruby-rack-mount, ruby-grape-entity, and ruby-grape for Hleb Valoshka.
• redir and tmate twice for Lucas Kanashiro.
• lxc to wheezy-backports for Christian Seiler.

Aprenda a criar repositórios de pacotes Debian

Neste post irei mostrar como configurar um repositório de pacotes Debian GNU/Linux usando dput + mini-dinstall + nginx. O Debian é um sistema operacional livre criado e mantido por um grupo independente de desenvolvedores espalhados pelo mundo, o projeto foi iniciado em 1993 e tem sido desenvolvido abertamente desde então sempre seguindo o espírito do projeto GNU.

Uma das características mais interessantes do Debian é o seu sistema de gerenciamento de pacotes.

“Um sistema de gerenciamento de pacotes é uma coleção de ferramentas que oferece um método automático para instalar, atualizar, configurar e remover pacotes de um sistema operacional. É tipicamente usado em sistemas operacionais tipo Unix, que consistem de centenas de pacotes distintos, para facilitar a identificação, instalação e atualização” (fonte: Wikipédia)

O sistema de gerenciamento de pacotes Debian chamado APT foi criado originalmente pelos desenvolvedores do projeto e lançado pela primeira vez em 9 Março de 1999 no Debian 2.1 (Slink) e conta hoje com um total de 44893 pacotes (44 mil !!!) em seus repositórios oficiais. Esta enorme quantidade de softwares disponíveis nos repositórios oficiais do projeto torna extremamente simple pesquisar, instalar, atualizar ou remover qualquer coisa do computador.

Os pacotes disponibilizados nos repositórios oficiais do Debian passam por um rígido controle de qualidade liderado pelos desenvolvedores oficiais do projeto e aberto à contribuição de qualquer pessoa. Então se você está empacotando algo veja o Guia do Novo Mantenedor Debian para saber como incluir isso nos repositórios oficiais.

Mas se isso não for possível ou se você não puder esperar o pacote entrar no Debian seja por qual motivo for, então a solução será criar um repositório pessoal ou Private Package Archive (PPA). É isto que será descrito aqui a partir de uma solução proposta inicialmente por Stefano Zacchiroli em seu blog no post howto: uploading to people.d.o using dput.

Configurando o servidor para receber pacotes

• Sistema Operacional: Debian Wheezy
• Domínio: debian.joenio.me

Acesse o servidor via SSH e instale o mini-dinstall:

mini-dinstall is a tool for installing Debian packages into a personal APT repository

# apt-get install mini-dinstall

Crie o diretório onde os pacotes serão copiados:

$ mkdir -p ~/debian.joenio.me/mini-dinstall/incoming

Crie o arquivo ~/.mini-dinstall.conf:

[DEFAULT]
mail_on_success = false
incoming_permissions = 0750
architectures = all, i386, amd64
archive_style = flat
dynamic_reindex = 1
archivedir = /home/joenio/debian.joenio.me/
generate_release = 1
release_label = Joenio
release_description = Unofficial Debian packages maintained by Joenio Costa
verify_sigs = 0

Preparando a estação de trabalho para enviar pacotes

• Sistema Operacional: Debian Testing

Instale o dput em seu computador:

dput allows you to put one or more Debian packages into the archive

# apt-get install dput

Exporte sua chave pública GnuPG para para o arquivo signing.asc.

$ gpg --armor --output signing.asc --export D5609CBE

Copie o arquivo signing.asc para o servidor:

$ scp signing.asc debian.joenio.me:~/debian.joenio.me/

Crie arquivo ~/.dput.cf em seu $HOME:

[debian.joenio.me]
fqdn = debian.joenio.me
method = scp
login = *
incoming = ~/debian.joenio.me/mini-dinstall/incoming
allow_unsigned_uploads = 0
post_upload_command = ssh debian.joenio.me mini-dinstall -b && sign-remote --no-batch debian.joenio.me:debian.joenio.me/unstable/Release

Os pacotes serão assinados com a chave GnuPG ao serem enviados pelo dput ao servidor. Isso será feito pelo script sign-remote disponível aqui. Copie este arquivo em seu computador e adicione a localização dele ao $PATH, se não sabe do que estou falando leia isso.

O dput irá reclamar se a distribuição do pacote não for unstable, isto é definido no arquivo changelog do pacote, veja o capítulo 4 Required files under the debian directory no Guia do Novo Mantenedor Debian para entender como fazer isso.

Bem, aqui já temos tudo pronto para fazer upload dos nossos pacotes ao servidor, basta executar o seguinte comando:

$ dput debian.joenio.me <pacote>.changes

Isto irá copiar o pacote .deb e o arquivo .changes dele, irá atualizar os arquivos Sources e Packages com as informações do pacote e irá assinar o arquivo Release com a chave pública GnuPG.

Disponibilizando acesso ao repositório publicamente

Até aqui já temos um repositório com pacotes hospedados, mas sem acesso público aos pacotes ninguém poderá utilizá-lo. Para resolver isto vamos configurar acesso via HTTP usando o servidor web Nginx.

Instale o Nginx no servidor:

# apt-get install nginx

Crie o arquivo de configuração /etc/nginx/sites-available/debian.joenio.me:

server {
  server_name debian.joenio.me;
  access_log /var/log/nginx/debian.joenio.me.access.log;
  location / {
    root /home/joenio/debian.joenio.me;
    autoindex on;
  }
}

Habilite o arquivo de configuração e reinicie o serviço:

# cd /etc/nginx/sites-enabled/
# ln -s /etc/nginx/sites-available/debian.joenio.me
# service nginx restart

Com o Nginx configurado e rodando já temos o repositório e seus pacotes disponíveis no seguinte endereço:

• http://debian.joenio.me

Testando o repositório e instalando pacotes

Para usar este repositorio adicione as seguintes entradas no /etc/apt/sources.list do computador local:

deb http://debian.joenio.me unstable/
deb-src http://debian.joenio.me unstable/

Adicione a chave de assinatura ao banco de dados de segurança do APT:

# wget -O - http://debian.joenio.me/signing.asc | apt-key add -

Atualize a lista de pacotes e teste a instalação de algum pacote disponível no repositório:

# apt-get update
# apt-get install <pacote>

Pronto, temos o nosso próprio repositório de pacotes Debian disponível publicamente. Você pode agora publicar qualquer pacote que esteja criando e divulgar o endereço do repositório para que outros usuários Debian possam instalar os seus pacotes mais facilmente.

Na wiki do Debian em How to setup a Debian repository tem uma lista de ferramentas para configuração de repositórios Debian, não usei a maioria das ferramentas listadas lá, mas dentre elas o aptly parece uma boa opção para criar repositório pessoal de pacotes Debian e pode ser uma boa alternativa para o setup descrito aqui neste post.

Ressuscitando seu celular após falha em mudança de flash

Como sou muito curioso e meu instinto hacker as vezes fala mais alto, comecei a futucar cada vez mais profundamente meu celular e testar algumas loucuras nele.

Em uma dessas tentativas, mesmo após ter feito backup, meu celular ficava com tela toda preta e nem aparecia a logo da empresa fornecedora no meu celular Pensei logo: “Pronto, perdi meu celular. Ganhei um peso de papel bem caro”.

Eu modifiquei o boot.img dele por uma imagem possivelmente defeituosa, mas eu jurava que ainda assim eu conseguiria acessar o recovery e ao menos restaurar o backup. Me enganei!

Acessando o fórum XDA Developers eu pude encontrar MUITA documentação do que poderia ser feito para resolver e tentei de tudo, mas nada parecia funcionar e meu nervosismo pela minha falha tornou minha busca no fórum bem mais complicada.

Felizmente eu consegui recolocar a flash de fábrica de volta no meu celular, eu segui os passos abaixo.

Firmware

Baixe o firmware relacionado ao seu celular, o meu é Xperia M e você pode pegar ele aqui aqui, mas caso o seu seja outro, basta procurar. A extensão dele é “ftf” e o meu firmware tinha 710 MB de tamanho.

Flasthtools

Primeiro precisamos baixar ele! Apenas baixe e coloque em uma pasta a sua escolha.

Ele vai reclamar que precisa da biblioteca libusb. Que pode ser obtida aqui. Depois de baixar vamos compilar e instalar com os comandos abaixo:

# tar xjvf libusbx-1.0.18.tar.bz2

# cd libusbx-1.0.18

# ./configure

# make

# make install

# ldconfig

O Flashtools precisa das bibliotecas no /usr/lib, mas a instalação da lib coloca ela no /usr/local/lib. Você pode mover para o local desejado, eu preferi criar link simbólico:

# ln -s /usr/local/lib/libusb-1.0.so /usr/lib/libusbx-1.0.so
# ln -s /usr/local/lib/libusb-1.0.so.0.1.0 /usr/lib/libusbx-1.0.so.0.1.0

Coloque o firmaware obtido no passo anterior na pasta firmawares do Flashtools.

Pronto! Agora inicie o seu flashtools como root. Ele abrirá uma tela pedindo pra mover a pasta firmwares de onde você baixou para /root/.flashTool/firmwares. Pode mover, isso mesmo mover, pois caso ele perceba que ainda existe a pasta firmawares no local hoje você executa o flashtools ele não abrirá o software corretamente.

Ele vai abrir uma tela mais ou menos assim:

Clique no ícone do raio e selecione a opção “flashmode”:

Na próxima tela selecione o firmware desejado:

Caso queira mudar mais alguma coisa, fique a vontade, eu não precisei mudar nada Basta clicar no botão “Flash”.

Agora aparecerá uma tela como essa, solicitando que seu celular entre em flashmode:

No caso do meu celular, eu arranquei a bateira, recoloquei ela, coloquei o cabo USB e pressionei o botão de volume para baixo até que o software flashtool reconhece ele

Depois ele começou o envio da flash. Bastou e esperar e meu celular estava prontinho para eu recomeçar tudo de novo

Muito obrigado a Nicklas Van Dam, que me ajudou MUITO lá no fórum XDA

MuseScore 2.0 is great, try it!

A bit of context: two years ago I joined an undergraduate program in electroacoustic music composition at the Université de Montréal. Fortunately the faculty has decided to use mostly free software in the classes. They recently moved from Max/MSP to Pure Data to teach algorithmic composition. OpenMusic has been used for computer assisted composition classes. On acoustics classes, Sonic Visualiser is the recommended tool. For everything related to audio processing and sound synthesis we mainly use Python pyo library and Cecilia, both developed by the professor himself. Other many free softwares are used for building digital musical instruments in the courses: Arduino, SuperCollider, OpenCV, openFrameworks etc.

So far I touched two proprietary softwares for my classes. First it was Reaper, a sequencer which has been recently adopted in replacement of Pro Tools in some grades. Reaper has a less unfair distribution model compared to Pro Tools and despite being a closed piece of software it somewhat looks like a community-oriented project, being developed by a small team of free software enthusiasts. Being an amazing, complete and still lightweight DAW I hope it goes free some day in the future (I've read about this possibility somewhere in a forum that I can't find now). Anyway, after some months playing with Reaper I went back to Ardour. Because it's free, not because it's better (Reapper still seems unbeatable here).

The other was Finale, an alternative to MuseScore for music notation. I used it for three compositions mainly due to its playback capabilities. As a middle-aged music student I don't have the internal ear enough developed to listen orchestral textures, articulations and other details provided by expensive VST stuff. However, I found editing with Finale a pain in the ass. It's so bugged that I thought I were using a sort of alpha version. Basic editing is much more logical and elegant with MuseScore. After all, these first experiences with Finale didn't convince me that such realistic playbacks are adding any value to my music. I suspect that moving back to soundfonts or even composing with no playback at all will probably force me to exercice more critical/analytical listening whenever I need to understand the effects of a specific instrumental gesture and instrument combinations. So, I'm back to MuseScore. Not only because it's free, but also because it's better (at least for my current needs).

MuseScore has allowed me to easily edit music scores in a free operating system, using a small and not so powerful laptop. Unable to donate money to this amazing project I've been happily giving some time to it, by testing new releases, reporting issues, translating to portuguese and making available unofficial Debian packages while the current maintainer prepares the official one, which seems to be coming very soon. If you're a Finale/Sibelius user and don't strictly need that universe of orchestral VSTs samples for your music work, please give MuseScore a try. Have a quick look at its online handbook and in a few minutes you will be able to experience the real pleasure of music scoring using a computer. You can try different soundfonts, including the so nice Sonatina Symphonic Orchestra.

Below is a screenshot of MuseScore 2.0, which will be released very soon. You can download the RC version for your system in the MuseScore website.

MuseScore 2.0