Skip to content

Blog

Enterprise all your Twisted applications with Ldaptor

2000px-Twisted_Logo_(software).svg

We're proud to announce the release of 14.0.0 of Ldaptor, now a first party Twisted project! Ldaptor is an asynchronous LDAP (Lightweight Directory Access Protocol) client and server implementation written for Twisted in Python. The biggest change is that Ldaptor is now sponsored by Amplidata. Through them we were able to get development, bug fixes and Twisted first-party sponsorship back online. We now have continuous integration (CI) with a wide matrix of support for py26/py27/pypy using Twisted 12.1 to 14.0 (and trunk). We also have about 75% code coverage with unit testing! You can download 14.0.0 and other releases here: Ldaptor Github Releases For a full review of what has changed, feel free to take a look at our live documentation over at ReadTheDocs: Ldaptor Documentation and the Changelog itself. Backstory That is quite a jump from the last official release of 0.0.43 back in 2012 and from all the unofficial forks that have popped up to fill the void in between. Here is a bit of back story on how we got to where we are now. It was originally written and carried by Tommi Virtanen until 2012, since then Ldaptor was forked many ways to solve various problems and each distro of Linux and BSD had their own patches building up dust. In the spring of 2014, an internal project at Amplidata required an OpenLDAP client for their Twisted services and the only one that offered the most promise was Ldaptor. We got in touch with Tommi (tv42) and Glyph of Twisted to work out an arrangement where Amplidata would sponsor continued work on Twisted, Tommi would re-license Ldaptor under the MIT Expat License and it would be hosted as a first party library with Twisted. Since then we've consolidated the bug-fixes of other forks and distributions, improved the unit tests, cleaned up the code-base and managed to recover the PyPI Ldaptor entry. Once Travis was all green, we made our first release 14.0 (on Halloween) and are now seeing development picking up with pull requests for more tests and features! Usage and Example This particular example also includes how to connect to OpenLDAP with StartTLS. This particular feature is critical to Amplidata and there isn't any Ldaptor information about it. Now there is!


from OpenSSL import SSL from twisted.internet import reactor, defer, ssl from ldaptor.protocols.ldap import ldapclient, ldapsyntax, ldapconnector

class ServerTLSContext(ssl.DefaultOpenSSLContextFactory): def init(self, args, kw): kw['sslmethod'] = SSL.TLSv1_METHOD ssl.DefaultOpenSSLContextFactory.init(self, args, **kw)

@defer.inlineCallbacks def example(): serverip = '192.168.128.21' basedn = 'dc=example,dc=com' binddn = 'bjensen' bindpw = 'secret' ssl = True query = '(cn=*)' c = ldapconnector.LDAPClientCreator(reactor, ldapclient.LDAPClient) overrides = {basedn: (serverip, 389)} client = yield c.connect(basedn, overrides=overrides)

# do you want SSL/TLS, then you need to create a context for startTLS
if ssl:
    tls_ctx = ServerTLSContext(
        privateKeyFileName='your.key',
        certificateFileName='your.crt'
    )
    yield client.startTLS(tls_ctx)

yield client.bind(binddn, bindpw)
o = ldapsyntax.LDAPEntry(client, basedn)
results = yield o.search(filterText=query)
for entry in results:
    print entry

if name == 'main': df = example() df.addErrback(lambda err: err.printTraceback()) df.addCallback(lambda _: reactor.stop()) reactor.run()


The above should work as-is, but you'll need to change the IPs, basedn, binddn, certs and keys. If you don't need SSL/TLS, then just ssl to False and you should be ready to go!

VideocoreIV Glamor on your Raspberry Pi

RaspberryPi Logo

Running an X (Xorg) server on your Raspberry Pi is frustrating. You can either use the fbdev or fbturbo driver which will give an un-accelerated 2D environment with swrast 3D (OpenGL) all beating your poor RPi's CPU. Overclocking it will only help you so much which is a pity considering that there is another layer on the SoC that would be perfect for that but is now unused.

Enter the VideocoreIV (VC4) and Eric Anholt (formally of Intel, now of Broadcom), who are going to breath new life into the RPi. The idea is to offload the 2D rendering, via Glamor, to the VC4 with OpenGL calls. Since a OpenGL stack needs to exist, that means there will be a Direct Rendering Manager (DRM) Linux kernel module and Gallium/DRI module in Mesa.

This is happening now, here is the current status of support via the Piglit test-suite: skip 19102, fail 3866, pass 3146, crash 153, total 26267

While we skip a great deal of tests, they are because some extensions are not yet implemented. Of those implemented, almost half of the tests are passing! Wrapping up the crashed and failed tests will help add stability to the OpenGL stack that will lead to the eventual use of the VC4 as an everyday use of the RPi.

If you would like to follow along from home, and I suggest you do as there are updates almost every day, then you can see the progress for yourself! The end result, at least for me, is to see OpenMW running on the RPi with full OpenGL acceleration under X. Eric has left some low-hanging fruit for new and would be contributors to help out! I'll provide a check-list of things you'll need, best practices and ways that you can help too.

Phase 1: Setting up your RPi workbench You'll need an RPi, running the latest Raspbian "Jessie" release. Setup a 'dev' directory under your account. Optionally, but highly recommended is to set this dev directory up as an NFS mountpoint so that your compiling/working is not ruining your SD-card with constant writes.

Phase 2: Getting the VC4 kernel driver At this point, you'll need to git clone Eric's fork of the kernel and get familiar with building your own kernel. You'll need do the following:

  • sudo apt-get install git build-essential ncurses-dev
  • git clone -b vc4 --depth=10 https://github.com/anholt/linux.git
  • cd linux; zcat /proc/config.gz > .config; make oldconfig

The basic idea is that you are cloning his specific branch and not the whole tree, otherwise your just wasting time. You'll pull in your running kernel's config and bring it up to date with what is new in Eric's VC4 branch.

You'll also need to modify a few things in the .config file, first: CONFIG_CMA_SIZE_MBYTES=5 --> CONFIG_CMA_SIZE_MBYTES=64 This needs to be set to, at least, 64 since 5 is NOT enough for the VC4 kernel module. The VC4 DRM module on the other hand is best, currently at least, to be built into the kernel. You'll need to turn on DRM and VC4 like below.

Direct Rendering Manager

CONFIG_DRM=y CONFIG_DRM_KMS_HELPER=y CONFIG_DRM_KMS_FB_HELPER=y

CONFIG_DRM_LOAD_EDID_FIRMWARE is not set

CONFIG_DRM_GEM_CMA_HELPER=y CONFIG_DRM_KMS_CMA_HELPER=y

I2C encoder or helper chips

CONFIG_DRM_I2C_CH7006 is not set

CONFIG_DRM_I2C_SIL164 is not set

CONFIG_DRM_I2C_NXP_TDA998X is not set

CONFIG_DRM_UDL is not set

CONFIG_DRM_ARMADA is not set

CONFIG_DRM_RCAR_DU is not set

CONFIG_DRM_SHMOBILE is not set

CONFIG_DRM_PTN3460 is not set

CONFIG_DRM_VC4=y

If you don't wish to do this by hand, just give the 'make menuconfig' a spin and you'll work through the options available there. A this point, you should be ready to build your kernel. The first time around will take a while.

make; make modules; make modules_install

After it is finished, time to copy your newly baked kernel and copy it to /boot to be usable. If you screw up, and your RPi no longer boots, just insert your SD-card into a FAT readable machine and edit the config.txt file to point to the old kernel since we'll be moving it out-of-the-way first.

sudo mv /boot/kernel.img /boot/kernel.img.orig; sudo cp arch/arm/boot/zImage /boot/kernel.img; sync

At this point, cross your fingers and reboot your RPi. If it comes back online, check your dmesg for the following output:

[ 1.267449] BCM2708FB: allocated DMA memory 58400000 [ 1.272631] BCM2708FB: allocated DMA channel 0 @ f2007000 [ 1.298488] Console: switching to colour frame buffer device 200x75 [ 1.320303] uart-pl011 dev:f1: no DMA platform data [ 1.325906] vc-cma: Videocore CMA driver [ 1.330043] vc-cma: vc_cma_base = 0x00000000 [ 1.334843] vc-cma: vc_cma_size = 0x00000000 (0 MiB) [ 1.340424] vc-cma: vc_cma_initial = 0x00000000 (0 MiB) [ 1.346345] Initialized drm 1.1.0 20060810 [ 1.361836] vc4-drm vc4-drm.0: fb1: frame buffer device [ 1.367283] vc4-drm vc4-drm.0: registered panic notifier [ 1.372819] Initialized vc4 0.0.0 20140616 on minor 0

At this point, you are good to go! The glue between VC4 and userland is ready to be abused! Keep in mind, if there are updates on the linux side, git pull and rebuild your kernel and modules. Don't forget to install both! Thankfully this doesn't happen often as most of the work happens in Mesa.

Phase 3: Getting Mesa and building the VC4 Gallium DRM

This is pretty straight forward, if you know the right magic during configuration. We've (Eric and I) finally nailed with is necessary to build and get working.

  • sudo apt-get install xorg-dev libgl1-mesa-dev libgles1-mesa-dev libgles2-mesa-dev libglu1-mesa-dev xutils-dev xserver-xorg-dev
  • git clone --depth=10 git://anongit.freedesktop.org/mesa/mesa
  • cd mesa; autoreconf -v --install
  • ./autogen.sh \ --prefix=$HOME/prefix \ --with-gallium-drivers=vc4,swrast \ --enable-gles1 \ --enable-gles2 \ --with-egl-platforms=x11,drm \ --enable-glx-tls \ --enable-dri \ --with-dri-drivers=swrast \ --enable-shared-glapi \ --enable-texture-float \ --with-log-dir=/var/log \ --prefix=/usr \ --libdir=/usr/lib/arm-linux-gnueabihf
  • make; sudo make install

What is going on here is that you need to install some development headers so that you can build your Mesa drivers that will be used with Xorg. You might need to install more packages, if so, please notify me but these should work and configuration shouldn't fail. At the end of the configuration, it should give you an output of everything it will do during compilation. Yours should look like mine:

prefix: /usr exec_prefix: ${prefix} libdir: ${exec_prefix}/lib includedir: ${prefix}/include

OpenGL: yes (ES1: yes ES2: yes) OpenVG: no

OSMesa: no

DRI platform: drm DRI drivers: swrast DRI driver dir: ${libdir}/dri GLX: DRI-based

EGL: yes EGL platforms: x11 drm EGL drivers: builtin:egl_dri2

llvm: no

Gallium: yes

Shared libs: yes Static libs: no Shared-glapi: yes

CFLAGS: -g -O2 -Wall -std=c99 -Werror=implicit-function-declaration -Werror=missing-prototypes -fno-strict-aliasing -fno-builtin-memcmp CXXFLAGS: -g -O2 -Wall -fno-strict-aliasing -fno-builtin-memcmp Macros: -DUSE_EXTERNAL_DXTN_LIB=1 -D_GNU_SOURCE -DTEXTURE_FLOAT_ENABLED -DHAVE_DLOPEN -DHAVE_POSIX_MEMALIGN -DHAVE_LIBDRM -DGLX_USE_DRM -DHAVE_LIBUDEV -DGLX_INDIRECT_RENDERING -DGLX_DIRECT_RENDERING -DGLX_USE_TLS -DHAVE_ALIAS -DHAVE_DRI3 -DHAVE_MINCORE

PYTHON2: python2

Phase 4: Piglit testing

This is the part that will tell you that everything you've done up to this point hasn't been in vain. Piglit is a OpenGL test-suite that is very thorough. You'll use this when developing, to test against.

  • sudo apt-get install libwaffle-dev python-dev
  • git clone --depth=10 git://anongit.freedesktop.org/git/piglit
  • cd piglit; cmake -DPIGLIT_BUILD_GLES2_TESTS=TRUE .
  • PIGLIT_PLATFORM=gbm ./bin/shader_runner tests/shaders/glsl-algebraic-add-add-1.shader_test -auto -fbo

After running the individual test, you should get something like this:

PIGLIT: {"result": "pass" }

Don't mind the rather noisy MESA-LOADER issues, the RPi doesn't have a PCI bus. Congratulations, you're now utilizing your VC4! Happy hacking!

For additional output, debug information and trace level QIR calls, try running the former command like this:

PIGLIT_PLATFORM=gbm VC4_DEBUG=qir ./bin/shader_runner tests/shaders/glsl-algebraic-add-add-1.shader_test -auto -fbo

MESA-LOADER: malformed or no PCI ID MESA-LOADER: malformed or no PCI ID libEGL debug: Native platform type: drm (environment overwrite) libEGL debug: EGL search path is /usr/lib/arm-linux-gnueabihf/egl libEGL debug: added /usr/lib/arm-linux-gnueabihf/egl/egl_gallium.so to module array libEGL debug: added egl_dri2 to module array libEGL debug: dlopen(/usr/lib/arm-linux-gnueabihf/egl/egl_gallium.so) libEGL info: use DRM for display 0x16a1568 libEGL debug: EGL user error 0x3001 (EGL_NOT_INITIALIZED) in eglInitialize(no usable display)

libEGL debug: the best driver is DRI2 QIR: mov t2, u2 mov t3, u3 mov t4, u4 mov t5, u5 fadd t6, u0, t2 fadd t7, u1, t3 fadd t8, u1, t4 fadd t9, u1, t5 pack_colors t15, t6, t7, t8, t9 tlb_color null, t15 ... PIGLIT: {"result": "pass" }

Now on to the real work, as Eric describes it in his LJ article helping out with VC4:

Now the actual work: I've left some of the TGSI opcodes unfinished (SCS, DST, DPH, and XPD, for example), so the driver just aborts when a shader tries to use them. How they work is described in src/gallium/docs/source/tgsi.rst. The TGSI-to_QIR code is in vc4_program.c (where you'll find all the opcodes that are implemented currently), and vc4_qir.h has all the opcodes that are available to you and helpers for generating them. Once it's in QIR (which I think should have all the opcodes you need for this work), vc4_qpu_emit.c will turn the QIR into actual QPU code like you find described in the chip specs.

Phase 5: Bonus Round -- Modesetting Xorg driver

You're likely thinking, right... all this work and how do I get it into Xorg's driver seat? The answer is the 'modesetting' driver which acts as the glue to getting X talking with Glamor. Be warned though, there isn't anything usable yet. This might change though in a month or two so keep an eye on this. Who knows, glxgears might be just around the corner!

  • git clone git://people.freedesktop.org/~anholt/xf86-video-modesetting
  • ./configure --prefix=/usr
  • make; sudo make install

You'll need to create an xorg.conf file and run startx to see anything. However, be again warned, if you do this, the VC4 will be in an unknown state afterwards and likely need a reboot of your RPi.

sudo vim /etc/X11/xorg.conf

Section "Module" Load "glamoregl" Load "glx" EndSection

Section "Device" Identifier "Default screen" Driver "modesetting" # you can change this back to fbdev Option "ForceGallium" "True" Option "AccelMethod" "glamor" EndSection

Section "dri" Mode 0666 EndSection

As always, you should also keep up to date with what Eric has been up to here: http://anholt.livejournal.com/

This wraps up hacking the VC4 on your Raspberry Pi boot-camp and I hope it was useful enough to get you started in the right direction.

Development on the Raspberry Pi

RaspberryPi Logo

Now that I'm a proud owner of a Raspberry Pi, I've being really stressing the little guy. There is only but so much a ARMv6 processor, on an microSD with only 512MiB of ram can do, which means that compiling on such a machine is going to take a really long time.

Take for example OpenMW, currently it takes about 4 minutes on a quad-core i7 to compile. You're in for a treat on the Pi, it will take you at least a day, two days if you realize that half-way through the OOM Killer came through and killed your cc process. This is about the time you start wondering about various ways to improve the situation, such as a larger swap file or using zram.

At this point, I was wondering about other ways compiling binaries and packages for the Pi. There was cross-compiling, but then I would have to set up a full toolchain and recompile all the packages from scratch. That will have to be for another post though as it is another world. Another option is to try virtualizing the Pi and apparently QEMU gets us pretty darn close.

Let us begin with the preparation of our environment. We'll get the latest Raspbian release and clean up it for use in a QEMU environment.

# get the latest Raspbian: wget http://downloads.raspberrypi.org/raspbian_latest

create new target image of about 20GiB and mount it

dd bs=1 count=0 seek=20G if=/dev/zero of=raspbian.img sudo losetup -f --show raspbian.img

mount downloaded raspbian image and copy over

sudo losetup -f --show 2014-06-20-wheezy-raspbian.img sudo dd if=/dev/loop1 of=/dev/loop0 sudo losetup -d /dev/loop1

resize partition, edit one file, and umount

sudo parted /dev/loop0 resizepart 2 20GB

resize our filesystem

sudo partprobe /dev/loop0 sudo resize2fs /dev/loop0p2

editing out shared lib preloading so that it fully boots

sudo mount /dev/loop0p2 /mnt/image sudo sed -i s/'\/usr'/'#\/usr'/g /mnt/image/etc/ld.so.preload cp /mnt/image/boot/kernel.img ~/kernel.img # used later in the custom built qemu sync sudo losetup -d /dev/loop0

At this point, you have an image for QEMU to use but now you have to choose. There are two routes:

  • The default QEMU with only 256MiB of ram and a slightly different kernel than the Raspbian one because it uses a different CPU. The benefit here is that you don't have to do much, just apt-get install and you're on your way.
  • The modified QEMU, that gives you native RPi support so you can use your Raspbian kernel. It also has 512MiB of ram, like the RPi B+ which helps with compiling. The only drawback currently is the lack of networking.

First choice: # get a kernel: wget http://xecdesign.com/downloads/linux-qemu/kernel-qemu

fire up QEMU:

qemu-system-arm -kernel kernel-qemu -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1 rootfstype=ext4 rw" -hda raspbian.img

Second choice: # setup build environment, configure and compile sudo aptitude install libsdl-dev libfdt-dev git build-essential git clone git://github.com/Torlus/qemu.git -b rpi cd qemu ./configure --target-list="arm-softmmu arm-linux-user" --enable-sdl make -j8

remember that kernel.img we copied earlier? We can use it here!

run our newly made system

arm-softmmu/qemu-system-arm -M raspi -m 512 -sd ~/raspbian.img -kernel ~/kernel.img -append "earlyprintk loglevel=8 panic=120 keep_bootcon rootwait dma.dmachans=0x7f35 bcm2708_fb.fbwidth=1024 bcm2708_fb.fbheight=768 bcm2708.boardrev=0xf bcm2708.serial=0xcad0eedf smsc95xx.macaddr=B8:27:EB:D0:EE:DF sdhci-bcm2708.emmc_clock_freq=100000000 vc_mem.mem_base=0x1c000000 vc_mem.mem_size=0x20000000 dwc_otg.lpm_enable=0 kgdboc=ttyAMA0,115200 console=tty1 elevator=deadline rootwait root=/dev/mmcblk0p2 panic=1 rootfstype=ext4 rw" -serial stdio -device usb-kbd -device usb-mouse -usbdevice net

At this point, you should have a usable Raspbian environment which should be faster than the real deal. Perfect for compiling and creating packages for your Raspberry Pi.

Happy hacking!

WildMIDI 0.3.7 has been released

WildMIDI

We're still quite busy with developing 0.4 and thought it was best to backport our fixes into the 0.3 branch. As a result we ended up having an additional 2 releases in the 0.3 branch!

What's new in this release

We've added DOS support in the form of SoundBlaster 16 playback in the player. The other big surprise is MIDI type-2 support, which came as a by-product of fully supporting XMI files like those from The Legend of Kyrandia. The rest of the changes are further enhancements to our build system and bug-fixes.

What to look forward to...:

We've wrapped up XMI (Eye of the Beholder 3) and Id's MUS (Doom, Raptor) support (with change of frequency support). We also have some preliminary support for playback of HMI (Daggerfall, X-COM) and HMP (Descent, Abuse) files! We are also working with the community and sharing our results, in response, they (to yet be named) are going to re-license their research and code to be more compatible with WildMIDI! This will certainly make engine re-implementation work easier for those that wish to use the library playback music. This is all being worked on currently in the 0.4 branch.

If you are interested in helping out, please leave a comment or fork the code on github! We're always looking at supporting more MIDI like formats.

The full change log:

  • Plug a memory leak in case of broken midis.
  • Properly reset global state upon library shutdown.
  • Support for type-2 midi files.
  • Fix a possible crash in WildMidi_SetOption.
  • DOS port: Support for Sound Blaster output in player.
  • Uglify the library's private global variable and function names.
  • Build: Add option for a statically linked player.
  • Build: Add headers to project files. Use -fno-common flag.
  • Other small fixes/clean-ups.
  • Fix some portability issues.
  • Fix a double-free issue during library shutdown when several midis were alive.
  • Fix the invalid option checking in WildMidi_Init().
  • Fix the roundtempo option which had been broken since its invention in 0.2.3.5 (WM_MO_ROUNDTEMPO: was 0xA000 instead of 0x2000.)
  • Fix cfg files without a newline at the end weren't parsed correctly.
  • Handle cfg files with mac line-endings.
  • Refuse loading suspiciously long files.

Downloads Source and binaries can be found on our github release page: https://github.com/Mindwerks/wildmidi/releases

WildMIDI 0.3.5 has been released

WildMIDI

We have a brand new release on our 0.3 branch! The intent here is to be the bridge between the older 0.2 series and our feature rich 0.4 that is currently under development.

Downloads Source and binaries can be found on our github release page: https://github.com/Mindwerks/wildmidi/releases

New and Old Developers

After a few months of development, we've really brought WildMIDI a long way. We are rejoined by Chris Ison (Wildcode) who had taken a small break away from development. Another developer has also joined us from uHexen2, Ozkan Sezer, who has contributed quite a lot to the project. Welcome guys! If you want to help out, just create a github account, fork us and send pull requests. Other forms of help are also appreciated, tracking down bugs and new platforms for us to support.

What's new in this release

In this release, we've incorporated a lot of bug fixes coming from downstream, mostly Linux distributions and GStreamer. We also took the time to do some static analysis and abuse valgrind to help improve code quality. The biggest win here was that we have taken a 2 minute midi file that would have used nearly 16,000 reallocs and would blow through about 400MiB of memory to just 600 reallocs and 8 MiB of memory. Great stuff...

Change log:

  • Greatly reduced the heap usage (was a regression introduced in 0.2.3)
  • OpenAL support: This gains us OSX and other platforms that OpenAL supports for sound output!
  • DOS DJGPP support: This goes a long way to helping other DOS based applications like UHexen2.
  • MinGW support: This gains us win32 and win64 support using this toolchain.
  • OSS support: No longer uses mmap mode for better compatibility. This gains us NetBSD and OpenBSD support.
  • Worked around an invalid memory read found by valgrind when playing Beethoven's Fur Elise.rmi at 44100 Hz using the old MIDIA patch-set from 1994.
  • Build fixes for MSVC. Revised visibility attributes usage.
  • Fedora support: We are now ready to see this get pushed upstream to Fedora.
  • New portable file and path-name system to handle cross-platform support.
  • Support for Debian/kFreeBSD, Debian/Hurd and other Debian archs.
  • Many bug fixes, code clean-ups and cosmetic fixes.

What to look forward to...

We mentioned above about the 0.4 series, and we are now fully committed to new feature development. This also comes at a cost, the API will be changing. Chris will be working on "Live Events", Ozkan on SB16 support under DOS and myself getting XMI support in.

What will happen to 0.3? To be clear, our 0.3 branch will live on in maintenance mode and we'll only accept patches to fix bugs. This is save us time so we don't have to backport when doing changes in the 0.4 branch.

Overall, we're extremely proud of where we are with WildMIDI and we hope you are as well!

Introducing WildMIDI version 0.3.2

WildMIDI

After several years of silence, There is now a new version of WildMIDI! Chris "Wildcode" Ison seems to have fallen off the planet around February of 2012 and the bug reports and patches have been accumulating on his SourceForge page. I decided to dump his SVN repository to github and continue hacking where he left off. We are still very much 100% API/ABI compatible and new versions can be considered drop-in replacements. We will continue to be open to developers that wish to improve WildMIDI but keep in mind that our goal is to be small and fast. We also wish Chris the very best and want very much for him to rejoin the project.

What's new?

There have been a lot of changes since WildMIDI 0.2.3.5, mostly involving our new build system.

  • Switched build system away from autotools and now using cmake
  • We can build natively on: Linux, FreeBSD, Windows and OSX
  • We support GCC, Clang, XCode and Visual Studio
  • Pulled in all out-standing patches from SourceForge, Debian and FreeBSD
  • Compile without warnings using enforced -Werrors
  • Found and fixed additional bugs

What we need: We still need people with other platforms to test WildMIDI and file bug reports and patches so that we can support more platforms as well. This includes both the library and the player.

We are also looking into adding new features, such as: XMI playback support, SF2 and DLS support.

Downloads:

  • Win32 Binaries (exe, dll, lib): WildMIDI-0.3.2-win32.zip
  • OSX Binaries: Not currently available, they can be built from source but there is currently no sound-code for the wildmidi player. You can however dump music to a WAV file.
  • Source: wildmidi-0.3.2.tar.gz
  • Free Patches (GUS):freepats_20060219.orig.tar.gz (extract the 'freepats' directory to where you want and adjust the 'dir' in wildmidi.cfg to point to it)

For more information: If you wish to read more about the History and the Future of the library, please read our WildMIDI Project page.

Screenshots running on Windows and Linux:

Debian Package Maintenance

debian

Updated 2026-08-18

Debian's infrastructure has moved on since this was written. Alioth and git.debian.org were retired in 2018 and replaced by Salsa, Debian's GitLab instance, and the old git-* wrappers are now subcommands of gbp. The commands and URLs below have been updated to match.

I've been there before with Gentoo, as a developer, but times and situations have changed. Over the years I've been deeply involved with Ubuntu for my work and at its heart, Debian. I've become a Debian Developer to help get closer to the development process, package management and maintenance to do my share of the heavy lifting. The first thing I did was sign up at Salsa, the GitLab instance Debian uses for collaboration. Create an account and request guest access; you can then be added to the relevant packaging team. At this point, it is also a good idea join a Debian mailing list and/or join IRC at irc.oftc.net and have a nice chat with the folks in #debian or their various sub-channels where you would like to help. It is important to have a sponsor/mentor that can help you out if you a problem. You will need them when getting accepted into the Debian community in order to start working. If you want to help but don't know where to begin, have a look at the "Work-Needing and Prospective Packages" page. There are a few packages up for adoption, orphaned and others looking for someone to take over. Go to the appropriate group and ask about supporting your package of choice. I am currently interested in introducing a new bit of software into Debian. If accepted, it will eventually show up in Ubuntu which is an added bonus. Once your Salsa account is set up, upload your public key (SSH access) in your account page and you should now have ssh/git access to salsa.debian.org, which gets you on your way. Once signed in, set your .gitconfig with your name and email address:

[user]
    name = Bret Curtis
    email = [email protected]

From there, we should create your new git repository. On Salsa this is done through the web interface of the team you belong to, or with gbp create-remote-repo once you have a local repository:

gbp create-remote-repo --remote-url-pattern='[email protected]:fonts-team/%(pkg)s.git'

We can no do the rest on our local machine. We create our build environment and pull in our newly created git repo then push into that our first release.

sudo apt install debian-archive-keyring git-buildpackage pbuilder

# on Ubuntu
git-pbuilder create --distribution sid --mirror http://deb.debian.org/debian/ --debootstrapopts "--keyring=/usr/share/keyrings/debian-archive-keyring.gpg"
# on Debian
git-pbuilder create

Some extra commands that help to update the environment to the latest packages or add custom packages similar to having your own chroot envrionment.

git-pbuilder update
git-pbuilder login --save-after-login

From here on, we are pulling in our git repo, which should be empty. We do a gbp import-orig with a release of the project. This isn't the full repo history, just the release. We'll be asked questions about the import to validate the version number. This will create master and upstream branches. Each new release goes into the upstream branch with a tag. In master we will keep our /debian directory which should contain our control files.

git clone [email protected]:fonts-team/fonts-ebgaramond.git fonts-ebgaramond
gbp import-orig --pristine-tar ../v0.015.tar.gz
git commit -a -m 'Setting up the first release.'
git push --all

Once you have a working /debian control files in place, let's try to get it to build and produce Debian packages. We run lintian afterwards to try to catch any errors in our control files.

export DEB_BUILD_OPTIONS="parallel=8" # or how many cores you wish to use
gbp buildpackage --git-ignore-new --git-pbuilder
lintian --pedantic -I

From here on, talk to your mentor and when you think you are ready, it is then time to "Intent to Package" (ITP) and file a bug report. Make sure you can send email first with:

mail -s Testing_1 [email protected] < /dev/null

Should you get this back in your email, then are good to go! Use reportbug and follow through with the instructions with your ITP.

#On Ubuntu
reportbug --email [email protected] -B debian wnpp

#On Debian
reportbug --email [email protected] wnpp

This will get you on your way to maintaining your own package. This will file a bug and put it on the mailing list so that the appropriate people see this. Prepare for push-back and questions, this is normal. They will give advice to make sure your package is ready to be added to Debian. Good luck! Update: 20141024 - When there is a new release, here are the additional steps. First clone the repo if you don't already done so, then setup your branches.

git clone [email protected]:fonts-team/fonts-ebgaramond.git fonts-ebgaramond
cd fonts-ebgaramond
git branch -a
* master
  remotes/origin/HEAD -> origin/master
  remotes/origin/debian
  remotes/origin/master
  remotes/origin/upstream
git checkout -t origin/debian
git checkout -t origin/upstream
git branch -a
  debian
  master
* upstream
  remotes/origin/HEAD -> origin/master
  remotes/origin/debian
  remotes/origin/master
  remotes/origin/upstream
git checkout master
gbp import-orig --pristine-tar ../v0.015d.tar.gz

Here you follow prompts, then when finished, give it a spin with building. Be sure to update your changelog and any version related issues.


Last updated 18 August 2026: refreshed for Salsa, which replaced Alioth and git.debian.org in 2018, and for the gbp subcommands that superseded the old git-import-orig and git-buildpackage wrappers. The original post dates from 23 October 2013, with a follow-up on 24 October 2014.

Wireless BCM4312 with the 3.10, 3.11, 3.12 and 3.13 kernels

Broadcom Chipset BCM4312

The hybrid driver from Broadcom was updated again in September (2013) with all the previous patches and a few other fixes as well. This brings them up to support linux kernel 3.9, which is very much welcome.

Sadly it breaks again with >= 3.10 with many warnings and errors which isn't good considering that Ubuntu (13.10) Saucy Salamander is about to be released.

We do have a patch for you though that gets us working again up to the 3.11 kernel.

Chipsets supported by "Broadcom’s IEEE 802.11a/b/g/n hybrid Linux® device driver" are: BCM4311, BCM4312, BCM4313, BCM4321, BCM4322, BCM43224, BCM43225, BCM43227 and BCM43228.

The error you will run into:

bcurtis@Aria:~/Workspace/wl.orig$ make KBUILD_NOPEDANTIC=1 make -C /lib/modules/uname -r/build M=pwd make[1]: Entering directory /usr/src/linux-headers-3.11.0-11-generic' CFG80211 API is prefered for this kernel version Using CFG80211 API LD /home/bcurtis/Workspace/wl.orig/built-in.o CC /home/bcurtis/Workspace/wl.orig/src/shared/linux\_osl.o CC /home/bcurtis/Workspace/wl.orig/src/wl/sys/wl\_linux.o /home/bcurtis/Workspace/wl.orig/src/wl/sys/wl\_linux.c: In function ‘wl\_tkip\_printstats’: /home/bcurtis/Workspace/wl.orig/src/wl/sys/wl\_linux.c:3246:7: warning: passing argument 1 of ‘wl->tkipmodops->print\_stats’ from incompatible pointer type wl->tkip\_bcast\_data); ^ /home/bcurtis/Workspace/wl.orig/src/wl/sys/wl\_linux.c:3246:7: note: expected ‘struct seq\_file \*’ but argument is of type ‘char \*’ /home/bcurtis/Workspace/wl.orig/src/wl/sys/wl\_linux.c:3249:4: warning: passing argument 1 of ‘wl->tkipmodops->print\_stats’ from incompatible pointer type wl->tkipmodops->print\_stats(debug\_buf, wl->tkip\_ucast\_data); ^ /home/bcurtis/Workspace/wl.orig/src/wl/sys/wl\_linux.c:3249:4: note: expected ‘struct seq\_file \*’ but argument is of type ‘char \*’ /home/bcurtis/Workspace/wl.orig/src/wl/sys/wl\_linux.c: In function ‘wl\_reg\_proc\_entry’: /home/bcurtis/Workspace/wl.orig/src/wl/sys/wl\_linux.c:3470:2: error: implicit declaration of function ‘create\_proc\_entry’ [-Werror=implicit-function-declaration] if ((wl->proc\_entry = create\_proc\_entry(tmp, 0644, NULL)) == NULL) { ^ /home/bcurtis/Workspace/wl.orig/src/wl/sys/wl\_linux.c:3470:22: warning: assignment makes pointer from integer without a cast if ((wl->proc\_entry = create\_proc\_entry(tmp, 0644, NULL)) == NULL) { ^ /home/bcurtis/Workspace/wl.orig/src/wl/sys/wl\_linux.c:3475:16: error: dereferencing pointer to incomplete type wl->proc\_entry->read\_proc = wl\_proc\_read; ^ /home/bcurtis/Workspace/wl.orig/src/wl/sys/wl\_linux.c:3476:16: error: dereferencing pointer to incomplete type wl->proc\_entry->write\_proc = wl\_proc\_write; ^ /home/bcurtis/Workspace/wl.orig/src/wl/sys/wl\_linux.c:3477:16: error: dereferencing pointer to incomplete type wl->proc\_entry->data = wl; ^ cc1: some warnings being treated as errors make[2]: \*\*\* Error 1 make[1]: \*\*\* Error 2 make[1]: Leaving directory/usr/src/linux-headers-3.11.0-11-generic' make: *** Error 2

The rundown:

  1. Download the 32 or 64-bit version: http://www.broadcom.com/support/802.11/linux_sta.php
  2. Download the patch: wl_3.10.patch
  3. Extract the sources: cd ~/Downloads; mkdir -p wl; cd wl; tar xf ../hybrid-*.tar.gz
  4. Patch and compile the sources: patch -p2 < ~/Downloads/wl_3.10.patch; make; sudo make install; sudo depmod; sudo modprobe wl

Give Ubuntu a few seconds after loading the "wl" kernel module, then eventually the Network Manager will start looking for wireless networks.

TL;DR: These patches are required for a working wl kernel module for the 3.10 and 3.11 kernel series.

Update (2014-04-22): Known to also work with Raring (14.04) that uses the 3.13 kernel.

Transcoding for mobiles

transcode-icon

There are times where I need a few films on my mobile for my son to watch. For my particular mobile "Galaxy Gio" the constraints for hardware acceleration is 360p as anything bigger will cause it to switch to software which is slower and sometimes a slideshow. For the uninitiated, 360p means either 480x360 at 4:3 ratio (your old TV) or 640x360 at 16:9 ratio (your HD TV).

First we need to cleanup our file names and sometimes 'rename' doesn't work properly. We list all files and separate them by commas and then parse line by line to first remove everything between () and then everything between []. We then move from the old name to the new name. File name cleanup: IFS=","; files=`ls --format commas | tr -d '\040\011\012\015'`; for file in $files; do newfile=`echo $file| sed 's/(.*)//' | sed 's/\[.*\]_//' | sed 's/_\[.*\]//' `; mv $file $newfile; done The code above will try to remove spaces, and everything between () and [].

ls | while read -r FILE do mv -v "$FILE" `echo $FILE | tr ' ' '_' | tr -d '[{}(),\!]' | tr -d "\'" | tr '' '' | sed 's/_-_/_/g'` done The above code will convert all your spaces to underscores, remove characters like {}(),! and convert the filename to lowercase.

ls | while read -r FILE do mv -v "$FILE" `echo $FILE | tr ' ' '_' ` done The above just converts all spaces to underscores.

Mass encoding: IFS=', '; set +x; for x in `ls --format commas | tr -d '\040\011\012\015'`; do height=`mediainfo $x | egrep -i "Height " | cut -d":" -f2 | sed 's/[^0-9]*//g'`; if [ $height -gt 360 ]; then name=`echo $x | sed 's/[^0-9]*//g'`; avconv -i $x -vcodec mpeg4 -acodec copy -scodec copy -threads 4 -vb 800000 -vf scale=640:360 series_name_${name}_SD.mkv; fi done

This last bit will go through your directory looking for media files who's height is greater than 360 and then transcode it to a lower resolution MKV file. Feel free to change MKV to AVI or whatever avconv supports as a container format. This keeps the original music, fonts and subtitles intact. By lowering the resolution, we match the screen resolution of the mobile and allow for hardware acceleration which will spare your battery. This has been incredibly handy during long flights with the kids.

Update: Convert to xvid that converts the audio track to mp3, this helps with playback on low end hardware DVD players as well as mobiles. avconv -i something.HDTV.mkv -vcodec mpeg4 -vtag xvid -acodec libmp3lame -ab 128k -ac 2 -ar 44100 -scodec copy -threads 4 -vb 800000 -vf scale=640:360 something.HDTV.avi

For aac (better quality and to preserve 5.1 audio): avconv -i something.HDTV.mkv -vcodec mpeg4 -vtag xvid -acodec aac -ab 192k -ar 44100 -strict expiremental -scodec copy -threads 4 -vb 800000 -vf scale=640:360 something.HDTV.avi

Should you need to adjust the aspect ratio and/or scale down, there are tools available to help. Aspect Ratio Calculator avconv -i something.HDTV.mkv -vcodec mpeg4 -acodec aac -ab 192k -ar 44100 -strict expiremental -scodec copy -threads 4 -vb 800000 -aspect 45:22 -vf scale=640:313 something.HDTV.mkv

Worldsynth 0.11.0 released

Rivers

Worldsynth version 0.11.0 is released and can be found on github.

In this release we've added an additional algorithm for heightmap generation based on Ken Perlin's work in noise generation. We decoupled the sea-level to be configurable based on percentage, which in addition to masks we can now create islands. You can also save your world and open it up later since we use pytables to store our settings, metadata and our data in an open hdf5 format. You can also export your heightmap in 16-bit PNG greyscale or even import from a wide array of images formats as a heightmap. Importing from an image creates a 16-bit precision greyscale heightmap. In addition to this, one commenter ask about Python3 support, well now you have it.

The immediate future of Worldsynth has the following: fluvial erosion, better river erosion, better river flow/snaking and editing properties of world.

Here are some screenshots of some of the latest features:

noise

options

island

seaLevel

elevation