Back to Hastur

Table of Contents

[MythTV][63][?][63] Backend

Work around authentication failure bug

Manifests as failure of first authentication. All subsequent auths succeed.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=10604

Workaround is to force ID mapping in /etc/samba/smb.conf: 

idmap config * : range = 1000-1999999

Name Mangling

  • Samba mangles names with question marks in them by default
  • Fix with "mangled names = no" at share scope

OS X ACL permissions problem

Possible workaround is to set "acl check permissions = no" in smb.conf global section

2007-10-10

Get xmltv working

UK TV channels: http://www.bbc.co.uk/reception/digitaltv/index.shtml

Attempt another method

Bah, takes too long.

Method 3

Abandoned until later.

ToDo

Transcode to format for AppleTV

DVD ripping

2007-11-08 

# apt-get install dvdrip subtitleripper xvid4conf ogmtools
# apt-get install libdvdcss acidrip gpac vobcopy dvdbackup

CD ripping

2007-12-06 

# apt-get install abcde

It's convenient to use the laptop drive to rip. Perhaps consider creating an iso to encode from using abcde.

2008-04-08 

# apt-get install vorbisgain

Media Centre Groups

# groupadd -g 3001 music
# groupadd -g 3001 videos
# groupadd -g

Media Centre ACLs

Users

users.sh 

useradd -u 3000 -s /usr/sbin/nologin -c "Media Centre" -d "/nonexistent" media-centre
groupadd --gid 3001 mc-music
groupadd mc-videos
groupadd mc-movies
groupadd mc-photos
groupadd mc-tv
groupadd mcadm-music
groupadd mcadm-videos
groupadd mcadm-movies
groupadd mcadm-photos
groupadd mcadm-tv

Permissions

permissions.sh 

#!/bin/sh
MEDIA_ROOT=.
# Currently videos == movies, and I'm not using tv
MEDIA_TYPES="music movies photos"

# Get ACL for media type
#   rwx for user and mcadm-foo
#   r-- for mc-foo
#   --- for others
#   children inherit the same permissions
function getmediafacl
{
cat <<END
user::rwx
group::---
group:mcadm-$1:rwx
group:mc-$1:r-x
mask::rwx
other::---
default:user::rwx
default:group::r-x
default:group:mcadm-$1:rwx
default:group:mc-$1:r-x
defaultmask:rwx
default:other::---
END
}

for TYPE in $MEDIA_TYPES ; do
    getmediafacl $TYPE | setfacl -R --set-file=- $MEDIA_ROOT/$TYPE
    # Fix execute perms on non-directories
    find $MEDIA_ROOT/$TYPE ! -type d -print0 | xargs -0 chmod a-x
done

Fix Group Membership

Add daemons that need to index media files to groups 

usermod -a -G mc-movies,mc-photos,mc-music,mc-videos mediatomb
usermod -a -G mc-movies,mc-photos,mc-music,mc-videos mythtv

Streaming Media

apt-get install liblame-dev libogg-dev libvorbis-dev libsndfile1-dev
./configure

Hmm, can't remember what the hell I was installing here.

iTunes

# apt-get install mt-daapd
# vim /etc/mt-daapd.conf
[general]
web_root = /usr/share/mt-daapd/admin-root
port = 3689
admin_pw = kelthar
db_type = sqlite3
db_parms = /mnt/md3/media/music/metadata
mp3_dir = /mnt/md3/media/music/library
servername = Hastur
runas = mt-daapd
playlist = /mnt/md3/media/music/metadata/mt-daapd.playlist
extensions = .mp3,.m4a,.m4p,.ogg,.flac,.mpc
process_m3u = 1
scan_type = 2
compress = 1
[plugins]
plugin_dir = /usr/lib/mt-daapd/plugins
plugins = rsp.so,ssc-ffmpeg.so

EOF
#

Patch the typo make

WTF?? was I half asleep when I was doing this?

UPnP

See also MediaCentre.UPnP

Moved AppleTV to Frontrow with MediaCloud UPnP client plugin

MediaTomb Installation

Install 0.11 deb for amd64 

# wget http://apt.mediatomb.cc/key.asc -O- -q | sudo apt-key add -
# echo "# mediatomb
deb http://apt.mediatomb.cc/ etch main
" >> /etc/apt/sources.list
# apt-get update
# apt-get install mediatomb

Transcode Audio

Transcode audio to mp3 for Nokia N770

mediatomb-transcode-audio script: MediaCentre.N770#AudioTranscoding

Modifications for OGG to MP3 transcoding to play on Nokia N770 

<map from="ogg" to="audio/ogg"/>
...
<treat mimetype="audio/ogg" as="ogg"/>
...
<transcode mimetype="audio/ogg" using="audio-common"/>

    <profiles>
      <profile name="audio-common" enabled="yes" type="external">
        <mimetype>audio/mpeg</mimetype>
        <accept-url>yes</accept-url>
        <first-resource>yes</first-resource>
        <accept-ogg-theora>no</accept-ogg-theora>
        <agent command="mediatomb-transcode-audio" arguments="%in %out"/>
        <buffer size="1048576" chunk-size="131072" fill-size="262144"/>
      </profile>

Transcode Video

For N770

mediatomb-transcode-video-n770 script: MediaCentre.N770#VideoTranscoding

Modifications for OGG to MP3 transcoding to play on Nokia N770 

<map from="ogg" to="audio/ogg"/>
...
<treat mimetype="audio/ogg" as="ogg"/>
...
<transcode mimetype="audio/ogg" using="audio-common"/>

    <profiles>
      <profile name="audio-common" enabled="yes" type="external">
        <mimetype>audio/mpeg</mimetype>
        <accept-url>yes</accept-url>
        <first-resource>yes</first-resource>
        <accept-ogg-theora>no</accept-ogg-theora>
        <agent command="mediatomb-transcode-audio" arguments="%in %out"/>
        <buffer size="1048576" chunk-size="131072" fill-size="262144"/>
      </profile>

minidlna

Replaces mediatomb. Can work in conjunction with bubbleupnpserver to maintain playlists on the server, rather than on the client.

Config file : /etc/minidlna.conf:

Ports : tcp8200, udp1900, udp55697

Permissions : usermod -a -G mc-movies,mc-tv,mc-photos,mc-music,mc-videos minidlna

Config

media_dir=A,/export/mediasys/media/music/library
media_dir=V,/export/mediasys/media/movies/library
media_dir=V,/export/mediasys/media/tvshows/library
media_dir=A,/export/mediasys/media/photos/library
db_dir=/mnt/md6-media/metadata/minidlna
port=8200

Quotas

Enable quotas 

# modprobe quota_v2
#
  • add grpquota to /etc/fstab
  • edquota /mnt/md3

Currently getting "XFS: unknown mount option [grpquota]"

BitTorrent

Install rtorrent and moblock

# apt-get install rtorrent moblock

Configure Moblock

Moblock needs a number of NetFilter modules. I have the following loaded: 

nfnetlink_queue
nfnetlink
xt_mark
nf_conntrack_ipv4
xt_state
nf_conntrack
xt_NFQUEUE
x_tables

Allow local network 

# vim /etc/moblock/blocklists.list

Comment out the iana-private list: 

www.bluetack.co.uk/config/iana-private.gz

Whitelist local network 

# vim /etc/moblock/moblock.conf

Add lines 

WHITE_IP_IN="192.168.1.0/24"
WHITE_IP_OUT="192.168.1.0/24"

Start moblock: 

/etc/init.d/moblock start

Configure rTorrent

Scheduled Downloads

libCurl race condition fix

Build with IPv6

Get

wget http://libtorrent.rakshasa.no/downloads/libtorrent-0.12.6.tar.gz
wget http://libtorrent.rakshasa.no/downloads/rtorrent-0.8.6.tar.gz
wget http://home.samfundet.no/~sesse/libtorrent-0.12.6-ipv6-07.patch
wget http://home.samfundet.no/~sesse/rtorrent-0.8.6-ipv6-07.patch

Extract and Patch

tar -xzvf libtorrent-0.12.6.tar.gz
patch -p0 < ../libtorrent-0.12.6-ipv6-07.patch
tar -xzvf rtorrent-0.8.6.tar.gz
patch -p0 < ../rtorrent-0.8.6-ipv6-07.patch

Fix broken libtool

  • Broken custom libtool scripts. Use default instead.

    sed -i -e "s/^LIBTOOL=./LIBTOOL='libtool'/" libtorrent-0.12.6/scripts/libtool.m4 sed -i -e "s/^LIBTOOL=./LIBTOOL='libtool'/" rtorrent-0.8.6/scripts/libtool.m4

Build and Install

libtorrent

cd libtorrent-0.12.6/
./autogen.sh --enable-ipv6 && ./configure --enable-ipv6 && make
sudo make install

rtorrent

cd rtorrent-0.8.6/
./autogen.sh --enable-ipv6 && ./configure --enable-ipv6 && make
sudo make install

Run

  • libtorrent is installed in /usr/local/lib so need to set LD_LIBRARY_PATH LD_LIBRARY_PATH=/usr/local/lib /usr/local/bin/rtorrent

Upgrade rtorrent

From 0.8.6 to 0.9.2.

Get

wget http://libtorrent.rakshasa.no/downloads/rtorrent-0.9.2.tar.gz
wget http://libtorrent.rakshasa.no/downloads/libtorrent-0.13.2.tar.gz
tar -xzvf libtorrent-0.13.2.tar.gz
tar -xzvf rtorrent-0.9.2.tar.gz

Build

cd libtorrent-0.13.2
./configure --enable-ipv6 && make && sudo make install
cd ../rtorrent-0.9.2
./configure --enable-ipv6 && make
cp /usr/local/bin/rtorrent{,-0.8.6}
make install

Update Configuration

  • max_open_sockets command became network.max_open_sockets

    mv ~/.rtorrent.rc ~/.rtorrent-0.8.rc cp ~/.rtorrent-0.8.rc ~/.rtorrent-0.9.rc sed -i -e "s/^max_open_sockets\W*=/network.&/" ~/.rtorrent-0.9.rc ln -sf ~/.rtorrent-0.9.rc ~/.rtorrent.rc

rtorrent Web Frontend

rtgui

  • Installed
  • Need to recompile rtorrent with xmlrpc.

Rebuild libtorrent

# update
cd ~/src/libtorrent
git checkout master
git pull
git checkout 0.13.1
# build
sudo apt-get install libcppunit-dev
./autogen.sh
./configure --enable-ipv6
make && sudo make install

Rebuild rtorrent with xmlrpc-c

# update
cd ~/src/rtorrent
git checkout master
git pull
git checkout 0.9.1
# build
sudo apt-get install libxmlrpc-c3-dev
./autogen.sh
./configure --enable-ipv6 --with-xmlrpc-c
make && sudo make install

BitTorrent Multi-User

rutorrent

# libapache2-mod-scgi already installed
cd ~/src
svn checkout http://rutorrent.googlecode.com/svn/trunk/ rutorrent
sudo cp -a rutorrent/rutorrent /var/www/

Created /var/www/fix-permissions.sh 

# fix permissions
RUTORRENT=/var/www/rutorrent
chown -R root:root $RUTORRENT/ &&
    chown -R www-data:www-data $RUTORRENT/share/ &&
    find $RUTORRENT/share/ -type d -exec chmod 775 {} \; &&
    find $RUTORRENT/share/ -type f -exec chmod 664 {} \;

Created /root/new-rutorrent-users.sh 

#!/bin/sh

USERS=$*
#PRETEND=echo

RUTORRENT_ROOT=/var/www/rutorrent

function add\_rutorrent\_user
{
    user=$1
    NEW\_USER\_DIR="${RUTORRENT_ROOT}/conf/users/$user"
    RPC_PORT=$(( $(id -u $user) + 4000 ))  # 5000...
    RPC_MOUNT="/RPC$(( $(id -u $user) - 998 ))"  # 2...

    echo -e "Adding rutorrent user $user\tRPC_PORT=$RPC_PORT\tRPC_MOUNT=$RPC_MOUNT"

    $PRETEND mkdir -p "$NEW\_USER\_DIR" \
    && $PRETEND cp "${RUTORRENT_ROOT}/conf"/*.ini "${RUTORRENT_ROOT}/conf"/*.php "$NEW\_USER\_DIR"

    # rewrite config.php
    cat "${RUTORRENT_ROOT}/conf/config.php" \
        | sed "s%\(\$scgi_port\>\s\*=\s\*\)[0-9]\+%\1$RPC_PORT%
               s%\(\$XMLRPCMountPoint\s\*=\s\*\)[^;]*;%\1\"$RPC_MOUNT\";%" \
        > "$NEW\_USER\_DIR"/config.php


}

for user in $USERS
do
    add\_rutorrent\_user $user
done

htpasswd

htpasswd -c /etc/rutorrent/htpasswd $user

$[Get Code]96

Defer Service Startup

  • Removed from sysvinit startup all services that depend on the array being present
  • mt-daapd, mediatomb, samba etc

sysv-rc-conf

Windows Virtualization

SNMP

Install Cacti

apt-get install cacti

Configure Cacti

firefox http://hastur/cacti

Reduce Syslog Noise

Live TV Streaming

Beets music tagger

  • Beets

    apt-get install python-setuptools python-mutagen

    cd beets-1.0b2

    python setup.py install

  • Give up

    apt-get remove python-setuptools python-mutagen

  • Try again with Stefano's Ubuntu packages

Postgres

Host configuration

  • postgresql.conf listen_addresses = '*'

  • pg_hba.conf # gnome: samenet ipv4, any ipv6 host all gnome 192.168.0.1/16 md5 host all gnome ::/0 md5

User configuration

# su postgres
$ createuser -P gnome
...

Private Git Repo

$ mkdir repos
$ mkdir repos/aproject.git
$ cd repos/aproject.git
$ git --bare init

rssdler

tor and tsocks

sudo vim /etc/tsocks.conf

$[Get Code]103

# We specify local as 127.0.0.0 - 127.191.255.255 because the
# Tor MAPADDRESS virtual IP range is the rest of net 127.
local = 127.0.0.0/255.128.0.0
local = 192.168.0.0/255.255.255.0
local = 10.0.0.0/255.0.0.0

# Tor daemon
server = 127.0.0.1
server_port = 9050

Logitech Media Server

Implements Multi Room Audio

github : https://github.com/Logitech/slimserver

Downloads

Stable : http://www.mysqueezebox.com/download

Nightly : http://downloads.slimdevices.com/nightly/?ver=7.9

Configure

  • Add squeezeboxserver to mc-music group and restart

    usermod -a -G mc-music squeezeboxserver /etc/init.d/logitechmediaserver restart

  • Drop iptables

  • http://hastur:9000

Local Music Folder : /export/mediasys/media/music/library

Local Playlist Folder : /export/mediasys/media/music/meta

  • Create mysqueezebox.com account (optional)

Trac

Under Apache and mod_wsgi: https://help.ubuntu.com/community/TracApacheModWsgi

With postgres backend: http://trac.edgewall.org/wiki/DatabaseBackend#Postgresql (Purged postgres 8.4, retained postgres 9.1)

Postgres database

# Create postgres database
sudo -u postgres createdb trac
# Add postgres user trac
sudo -u postgres createuser -P trac
sudo -u postgres psql
trac=> alter user trac with password 'PASSWORD'
trac=> GRANT ALL PRIVILEGES ON DATABASE trac to trac;
# Install python bindings
sudo apt-get install python-psycopg2

Configure permissions in pg_hba.conf: 

# Add near the top of the file since config priority is top-to-bottom (like iptables)
local   trac             trac                                   password

Trac itself

# Initialize
sudo trac-admin /var/www/trac initenv
# You will be asked for the project-name and the database connection.
> trac
> postgres://trac:PASSWORD@/trac?host=/var/run/postgresql

# Install Apache2, mod_wsgi:
sudo apt-get install apache2 libapache2-mod-wsgi
# Configure WSGI by running:
sudo trac-admin /var/www/trac deploy /var/www/trac
# which will create a /var/www/trac/cgi-bin/trac.wsgi file.
# Give Apache the required privileges:
sudo chown -R www-data /var/www/trac

Configure Apache by adding the following to /etc/apache2/sites-available/trac: 

WSGIScriptAlias /trac /var/www/trac/cgi-bin/trac.wsgi
<Directory /var/www/trac>
    WSGIApplicationGroup %{GLOBAL}
    Order deny,allow
    Allow from all
</Directory>

Deluge Bittorrent Server

Server config

sudo apt-get install deluged deluge-console
sudo su --shell /bin/bash --login debian-deluged
deluged -d -L debug  # test

Auth

/var/lib/deluged/.config/deluge/auth 

# FIXME: add to new-user script!
echo "user:pass:5" >> .config/deluge/auth

Autoadd

Client config

deluge-console
> connect localhost [user] [pass]

Flexget config

Move flexget to deluge's crontab.

Add deluge to mcadm-movies and mcadm-tv so deluge can move completed files into place.

Flexget sets deluge's torrent location, in-progress download location and completed move location for each torrent file.

/var/lib/deluged/.flexget/config.yml 

templates:
  global:
    deluge:
      username: flexget
      password: "somepassword"

  deluge_tv:
    download: /var/lib/deluged/torrents/tvshows
    deluge:
      label: "flexget-tvshows"
      movedone: /mnt/md6-media/tvshows/new
      path: /mnt/md6-media/incomplete

  deluge_movie:
    download: /var/lib/deluged/torrents/movies
    deluge:
      label: "flexget-movies"
      movedone: /mnt/md6-media/movies/new
      path: /mnt/md6-media/incomplete

squid-deb-proxy

Deploy as a replacement for apt-cacher-ng.

Server

apt-get install squid-deb-proxy
# allow cognomen repo
cat > /etc/squid-deb-proxy/mirror-dstdomain.acl.d/20-cognomen
cognomen.co.uk
^D
# allow raspbian and osmc repos
cat > /etc/squid-deb-proxy/mirror-dstdomain.acl.d/30-osmc
mirrordirector.raspbian.org
apt.osmc.tv
^D
# update iptables
echo "8000 squid-deb-proxy" >> /etc/iptables/ports\_in\_tcp_allow

Allow access and caching of unofficial repos in /etc/squid-deb-proxy/squid-deb-proxy.conf: 

#http_access deny !to_archive_mirrors
http_access allow !to_archive_mirrors

# don't cache domains not listed in the mirrors file
# uncomment the third and fourth line to cache any unlisted domains
#cache deny !to_archive_mirrors
cache allow !to_archive_mirrors

Client

/etc/apt/apt.conf.d/50apt-proxy 

Acquire {
  Retries "0";
  HTTP { Proxy "http://hastur:8000"; };
};

NUT and UPS

https://thehomeserverhandbook.com/2012/02/28/monitoring_ups/

NUT configuration

Set startup mode in /etc/nut/nut.conf 

MODE=netserver

Add UPS to /etc/nut/ups.conf 

[eaton]
driver = usbhid-ups
port = auto
desc = "Eaton Eclipse ECO 1200"

Configure upsd in /etc/nut/upsd.conf 

LISTEN 0.0.0.0

Create upsd user credentials in /etc/nut/upsd.users, one for local monitor, one for home automation. 

[upsmon]
password = blah
upsmon master

[pixie]
password = foo
upsmon slave

Local upsmon client config in /etc/nut/upsmon.conf 

MONITOR eaton@localhost 1 upsmon blah master

Fix incorrect Debian permissions

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=721600

USB device rules need higher priority to set group ownership to nut: 

sudo cp /lib/udev/rules.d/52-nut-usbups.rules /etc/udev/rules.d/62-nut-usbups.rules

Log

2010-05-13

  • Set "acl check permissions = no"

2010-06-04

  • Implement user mediasys directories
  • Add House smb share

2010-06-14

  • Workaround libcurl race condition crashing rtorrent

2010-07-08

  • Fix Samba name mangling with "mangled names = no"

2010-07-13

  • Aborted installation of beets

2010-12-10

  • Installed updated rtorrent with IPv6

2010-12-13

  • Tweak logging of snmpd

2011-04-07

  • Install postgres for CV

2011-11-11

  • Add systems group

groupadd -g 3012 systems

  • Apply ACL for md6-media/systems

2013-10-22

  • Update libtorrent to 0.13.2 and rtorrent to 0.9.2

2014-02-05

  • Install rtgui. (unconfigured)
  • Built and installed libtorrent (branch 0.13.1 == libtorrent.so.17.0.4)
  • Built and installed rtorrent (branch 0.9.1 == rtorrent-0.9.3) to /usr/local/

2016-03-31

  • Try out deluged since rtorrent keeps crashing in res_send.c
  • Replace mediatomb with minidlna

2016-05-10

  • Remove apt-cacher-ng
  • Deploy squid-deb-proxy

2017-01-07

  • NUT

SideBar