Tag: linux

(perl-)rename – a must-have!

https://metacpan.org/pod/release/PEDERST/rename-1.9/bin/rename.PL

^ This!

Ever did some stupid for i in *.foo; do someconvert $i ${i}.bar; done , and hated the double “.foo.bar” extension? Or started naming files “1-foo“, “2-bar“, reached 10, and had to pad all the filenames with a zero? Hated doing it by hand, or writing stupid bash scripts to do it? Just use rename!

If you call in next 15 minutes, you will get a free…

…yes, yes, i know, but it’s a really useful tool!

Here are some examples stolen from the documentation page:

 rename 's/\.flip$/.flop/' # rename *.flip to *.flop
 rename s/flip/flop/ # rename *flip* to *flop*
 rename 's/^s\.(.*)/$1.X/' # switch sccs filenames around
 rename 's/$/.orig/ */*.[ch]' # add .orig to source files in */
 rename 'y/A-Z/a-z/' # lowercase all filenames in .
 rename 'y/A-Z/a-z/ if -B' # same, but just binaries!

Enabling SSH on a fresh Raspbian Jessie image

This post is mostly a reminder for myself, because I will probably forget this (again) soon.

Sometimes you want to use a RaspberryPi as a headless machine, so first you download the newest Rasbian image, ‘dd’ it to the SD card, insert the SD card into your RaspberryPi, connect the ethernet and power cables, check the IP in your DHCP server logs, run ‘ssh <ip>‘ and..

..connection refused!

I have no idea why, but the ssh server is disabled by default (probably a security feature, since the default password is easy to guess, but still..). Then you start googling, and first couple of links all say that you need to connect the keyboard, display, log in localy, etc. etc. But you’re lazy, and want to do it directly on the PC when you write the image to the SD card. So the easiest way is, after you’re done with ‘dd’-ing, you mount the first partition (a smaller, FAT partition containing the boot files), and just create an empty ‘ssh’ file (with eg: touch ssh) in that folder.

If you also wish to enable wifi (which is enabled by default, but no configurations are stored), you also need to mount the second (ext4) partition, containing the full system – there you can edit ‘./etc/wpa_supplicant/wpa_supplicant.conf‘ file, and add your wireless configuration

After that, sync, umount, insert the card, connect all the cables, and it works!

Mounting VirtualBox VDI images (and similar formats)

Sometimes you have a .vdi (or .vmdk or similar) virtual machine disk image, and you wish to access files inside, without starting the virtual machine itself and copying files via network or some ‘guest additions’ tools provided by the virtualization software.

To do this, you need nbd support enabled in the kernel (and module inserted) and qemu installed (which is actually another virtualization software, but we need it for the qemu-nbd utility).

In some distributions, the nbd module is enabled by default, on some you have to enable it when compiling the kernel sources.

  │ Symbol: BLK_DEV_NBD [=m]
  │ Type  : tristate
  │ Prompt: Network block device support
  │   Location:
  │     -> Device Drivers
  │       -> Block devices (BLK_DEV [=y])
  │   Defined at drivers/block/Kconfig:276
  │   Depends on: BLK_DEV [=y] && NET [=y]

Due to some weird reasons, the nbd module by default supports zero (0) partitions inside a block device:

# modinfo nbd
filename:       /lib/modules/4.9.6-gentoo-r1/kernel/drivers/block/nbd.ko
license:        GPL
description:    Network Block Device
depends:        
intree:         Y
vermagic:       4.9.6-gentoo-r1 SMP mod_unload 
parm:           nbds_max:number of network block devices to initialize (default: 16) (int)
parm:           max_part:number of partitions per device (default: 0) (int)

So when inserting the module, set the parameter to a larger number (eg. 4 or 16 or whatever)

modprobe nbd max_part=16

After that, you use the qemu-nbd utility to ‘connect’  the vdi image to the nbd block device (qemu-nbd actually works as a block device server), mount the partition, do stuff, umount the partition, and disconnect the qemu-nbd:

#connect the device:
qemu-nbd -c /dev/nbd0 /path/foo.vdi
#partitions appear as /dev/nbd0p1, /dev/nbd0p2...
mount /dev/nbd0p1 /mnt
#do stuff
umount /mnt
#and now disconnect the device:
qemu-nbd -d /dev/nbd0

 

Checking how much data still needs to be written when sync-ing on Linux

Sometimes on Linux, when you copy alot of data to a slow USB flash drive, the operation is “done” in a second, but when you umount the drive (or run ‘sync’) it takes forever to write the cache to the actual device.

So, how to check the status (how much time is left, how much does it still need to write)?

The answer hides in /proc/meminfo. Specifically the “Dirty” and “Writeback” lines in that virtual file. Very simplified, the “Dirty” value tells you how much data is waiting in cache, and the “Writeback” tells you how much data is being currently written to the device (chunk size).

You can check both values in semi-realtime with the following command:

watch "egrep -e '(Dirty:|Writeback:)' /proc/meminfo"

 

Fixing video tearing on Intel Broadwell graphics

I have a Thinkpad X250 laptop (Core i7 5600U CPU) with integrated graphics. When I watch video, in some cases (usually window mode + flash player) causes very noticable video tearing effect (almost like interlacing). It usually didn’t bother me (I mostly watch video fullscreen, where there is no tearing).

And the fix? More obvious then i though. Create a file /etc/X11/xorg.conf.d/15-intel.conf with this inside:

 Section "Device"
   Identifier  "Intel Graphics"
   Driver      "intel"
   Option      "TearFree"    "true"
EndSection

Yeah, who would have thought I had to enable the "TearFree" option…

Copying a file increasing in size

This is a quick hack, for when I used udpxrec (part of udpxy) to record an IPTV stream to an mpeg file, but instead of saving it to a network share (to watch it with some delay on my OpenElec box), i saved it to a local drive (by mistake). So here was a file, gradually increasing in size, which i wanted on my network drive (to start watching before the actual show/recording is finished). cp of course wont work, since it stops when it detects the end of file (does not detect new data being added, and wait for it), so you need to use something else.

When you think of a file with data being appended at the end, the first thought is “tail -f” (-f = follow and print the data being appended). Since tail only prints the last few lines (or bytes), you need to set it to output from the beginning with the “-c +0” (output bytes starting at the zero-th byte). I also pipe it through pv to follow the progress and copy rate.

So the command is:

tail -f -c +0 /path/source.mpg | pv > /destinationpath/destination.mpg