-
Notifications
You must be signed in to change notification settings - Fork 19
Whats new with Shoes 3.2 for Developers
For developers, there are many changes and you may not like them. I didn't either but it was the right thing to do. Most of this document is Linux centric with notes and asides for OSX and Windows.
Back when Ruby went to 1.9.3, Shoes compiled against 1.9.3 started to throw up error messages on the Linux terminal window. Mostly harmless (perhaps), but really annoying. That's what I wanted to fix first with Shoes and it lead to many changes. The problem was when Shoes asks ruby to initialize, ruby starts loading files (require) and it was hard coded to load from where Ruby was installed, not from Shoe's ruby copy. That cause encoding .so libs and some gem handling to be loaded from there and not from Shoes. That also caused some mostly hidden confusion with gem handling internals. There's another problem with - you can't copy a Ruby (Shoes) to another machine unless it already had a Ruby installed that would supply the internal requires. A Shoes linked to /home/ccoupe/.rvm/rubyies/ruby-1.9.3/ is not likely to work if your name isn't ccoupe and you use rvm. That explains why Linux .runs never worked reliably. (It's also happens on OSX but the error messages are hidden).
There is an option when you build Ruby to disable that and load from where the caller lives. That's --enable-load-relative, but it's not used often in my checks of Linux Rubies. You can't depend on your installed Ruby to be compiled for what Shoes needs. To distribute Shoes binaries, you have to compile your own Ruby, link to it and use it to copy from when when building a Shoes app/distribution.
There was one more problem. That hard coded place that ruby init looks for is 'lib/ruby/1.9.3' and Shoes 3.1 (and 3.0) had the ruby files in ruby/1.9.3 (or something like that). This used to get patched up in cache.rb when Shoes starts up but all this happens before one line of Shoes gets a chance to run. So files were moved around to be consistent with what Ruby wants. Demands is a better word. In fact there was a lot of cache.rb that attempted to fool Ruby instead of work with Ruby. I'll probably forget to write up what happens when compiling Shoes extensions like ftsearch and chipmunk.
I know some of you will think I'm wrong about having to build your own ruby. Perhaps you are more clever than I and you can bend Ruby C to your will.
Despite what you might hear, Ruby is fairly easy to build from source. What you might not want to do is install it over the top of your Ruby(s). Sadly, if the make install isn't run that Ruby isn't set up run. Not my rules. Of course if you're cross compiling an arm Ruby or Windows ruby you can even attempt to install it on a Linux box (without some magic - see below).
Once I realized what Ruby 1.9.3/2.+ is doing I came up with something I call LOOSE_SHOES. This is a Shoes that will only build and run on your system. Only a small chance in hell it will work for anybody else. It doesn't have it's own copy of Ruby it links to what you used to compile with. It doesn't have copies of libjpeg and libgif and so on. That makes it really fast to compile and it's small. Since it uses your Ruby, it has access to the gems you've installed. It also uses symlinks to lib/shoes/lib so if you working on Shoes Ruby code, you don't have to compile Shoes to test it. I did write an installer task to copy it to ~/.shoes/federales and set up a menu entry (Linux). The rake files won't let you package or distribute a LOOSE_SHOES. I'm sure you can see why. Loose Shoes doesn't live in the sandbox.
What I wanted to do from the very beginning was use my Ubuntu (intel box) to build Shoes for Raspberry. I thought it might be something the 'pi' crowd would like. (you can build from source on a pi if you have some spare hours). I wanted to cross compile Shoes. Then I realized that even X86_64 and i686 Shoes would have to be cross compiled for Linux distribution. And I could leverage my earlier attempt at trying to cross compile the Windows Shoes.
I've spent a lot of time poking around the the Shoes rakefiles and the kindest thing I can say is they work. I tried to clean them up but now there are 5 platforms (loose) and 3 (tight) that only seem similar. To quote Larry the Cable Guy, "Lord, I apologize".
The default build is LOOSE Shoes. You still have to figure which packages to install for your Linux. To build a Shoes you can copy to someone else, You have to do a lot of work. It's not hard work but it does take time to get it set up. The rakefile has some new tasks with Shoes 3.2
You use linux:setup:xxxx to say you're cross compiling. Then rake build,clean and package use a different set of rakefiles. use the linux:setup:clean option to go back to normal. Even osx builds are done this way (on OSX)
You could set up Virtual Machines for each platform and then log in and build shoes there. I felt that was not an elegant way to build Shoes in an automated fashion. Before you tell me I'm wrong, just let me tell you that I have a cron job that runs once a week and builds all five platforms in under 5 minutes of wall time (and most of that is in Windows NSIS installer)
I run Ubuntu Linux. It and other Debian derivatives have 'deboostrap' which will populate a chroot with just enough of a Debian system to run apt-get. You have to login to the chroot to do anywork - like running apt-get update or building Ruby. Understanding is hard until the light bulb goes on and then you'll giggle with joy. Even better, I have a schroot version of chroot. You can read all about with some google searches or even my posts at the Shoes 3.2 blog
The raspberry pi chroot is a little different. You have to install qemu properly so that when in that chroot the arm code (raspbian linux) runs in the qemu emulator. Then debootstrap raspian and build your own ruby with --enable-load-relative. It's not fast, but it might be faster than the native pi and how often do you build need to build Ruby? Because it is slow, I don't run the Shoes build in the raspberry pi chroot. I just copy the libs in there and use a cross compile running on X86_64.
MinGW/Windows build doesn't have a chroot. It does have a directory like a chroot where everything is downloaded and cross compiled if needed. Gtk2 binaries are available MinGW - use them. Cross compling your own Ruby for Windows is a major pain in the ass and even harder if you want a full Ruby (openssl, gdbm) extensions. You don't have to do very often, once it's working.
I wish I had truly understood how deeply RbConfig affects Ruby and Shoes. I didn't. I just hacked around problems like all the shoesers before me. Take the rakefiles - When you cross compile which is what Shoes 3.2 is all about, you need to know about the Ruby that is running the rake. What you want to use is the RbConfig of the target ruby you're build Shoes for. Currently, it's a sucky suck hack and both the compiling (host) Ruby and the target Ruby should be the same version. Look at the extconf for ftsearch/chipmunk if you want to bathe in the suckage.
There's no reason a Ruby 1.8.7 should not be able to run rake and build a Shoes that uses Ruby 2.1.2. Well, there is a reason and it's RbConfig and the fact that Ruby doesn't want you to do cross compiling or help embed ruby in a C program.
I do have a scheme in mind that would simplify the rake files and extconf.rb file but there's a lot of bugs to get to first.
Gem handling is much different in Shoes 3.2. So far, there are no built in gems that come with Shoes. None. Ruby and gems evolve must faster than Shoes does. Shoes should not distribute old unmaintained gems like hpricot and the dancing required to compile and supply a sqlite3.so/dylib/dll is a high hurdle, to mix my metaphors. So, Shoes 3.2 doesn't supply copies of hrpicot,json & sqlite. It does supply a modern Gems (2.2.2) which brings a lot of goodness. You may have to update the gem code when you build the rubies in the chroots.
Gems that needed a C compiler were problematic in Shoes 2 & 3. Now they work - if you have the compiler and system libs needed. You may not need them. Gems can have pre-compiled binaries. I believe nokogiri has them for Windows for example. Luis Ravena's rakecompiler can cross compile binary gems. That's the easier way to create a Sqlite3 gem for Windows -- If I wanted to (I don't).
Gem jailbreak allows even a TIGHT_SHOES ruby to use gems installed on the system. In Shoes 3.2 the user can install serialport (or similar) if there're willing to install the development tools need. That task is easier then ever before.
No need to have Shoes.setup in your scripts. If you do, it will work much better because it doesn't use cache.rb problematic rewrites for compiling code.
Cobbler is a Shoes utility (pure ruby) the user can run to tailor their Shoes. It can list, search, install and remove gems. I've written about it on the Shoes 3.2 blog. It's more of a User level utility than a developers concern.
Shoes 3.2 does not use curl for download. It does not use Object C for OSX for download. It does not use the winhttp stuff. It uses Ruby open-uri. Yes it may be a tiny bit slower. Yes, the download if needed stubs and packager may never work again. I'm OK with the trade-off.
OSX has a LOOSE_SHOES and a TIGHT_SHOES variant in the rake files. Only Tight Shoes has received much attention. As of July 2014, they aren't that different. That should change, once I start fixing some OSX bugs -it's much easier to debug a LOOSE SHOES.
Building on OSX depends on having Homebrew installed. Expect some trouble getting all the dependencies worked out.
You also need Xcode and xcode command line tools from the App store. Free, the last time I looked. You could build for 10.8 using 10.9 by changing a few characters in the rakefiles but who cares?
The new packager is named app_packager.rb (the old one still exists and still won't work). It only packages a complete Shoes with your script. The result is 15MB to 20MB, depending on platform. On the helpful side it works and it has feed back as it downloads and unpacks from the website (which can be tailored in Cobbler).
When dealing with Linux, all Shoes will use Gem's Zip and Tar routines. Not minitar. For linux there is a game played to get the filepos set in the .run to where the tarball starts. Oh, I got rid of the symbolic links in the linux launch script - another 3.0 era hack - that might be traceable to me, too. Lord, I apologize if it was me.
OSX does not use the DMG binject (it's still there and it works). Instead the Shoes osx is a .tgz. Shoes uses the Gem::Zip and Tar routines instead of Minitar, just like Linux only simpler.
Packaging for Windows does require calling into Binject::Exe
Other than the intellectual challenge (which is high) don't expect much more in the packager. I might get rid of Minitar and the old packager.
Windows has two variants. Native widget and using Gtk2. I only test and distribute the Gtk2 variant. Threadings issues may occur more often with native widgets or so I think. Gtk2 on windows is themed and supposedly there is a theme more windows like. Ask me if I care.
The source code for gtk.c (and associated .h) files has #ifdef's for Gtk3. It mostly works on Linux and Windows. Might be something for Shoes 3.3 ? References (and code specific) to Ruby 1.8 and 1.9 have been removed, as much as possible. There's probably still some. I removed everything having to do with video - I kind of regret that. Might be a neat 3.3 feature if I can figure out a cross platform way. Then again, it was a badly borked hack - it should die.
I'm sure I've missed describing some things that have changed internally. I still hate those rake files. Never the less, that most of what you need to know about shoes internals for 3.2
Now you know why I call it a maintenance release. Other than working where Shoes 3.1 wouldn't, there's very little that's exciting or whiz bang new.