Amitai Schlair on Open Source and Project Maintenance | SCNA 2013
Hi it's Mike with UGtastic again. I'm here today at day two of SCNA 2013. Hi it's Mike with UGtastic again. I'm here today at day two of SCNA 2013. Meet Amitai Schlair, a volunteer on the NetBSD Foundation board, as he discusses the importance of portability and craftsmanship in Unix-like systems. Learn how NetBSD's focus on portability and driver abstraction has been a key factor in its success.
The Interviewer
Mike Hall
Interviewer, UGtastic
The Guest
Amitai Schlair
open source and project maintenance
Mike Hall
Interviewer, UGtastic
▶ Speaking
Hi it's Mike with UGtastic again. I'm here today at day two of SCNA 2013. Right now I'm sitting down with Amitai Schleyer who sits on the TNF. As you might know, the TNF is the net BSD. The board of the I can't even say myself, but it's a board of the net BSD organization or a foundation. TNF I said that beautifully so. Anyway, hi hi Amitai. Thank you very much for taking the time to sit down with me. Appreciate it.
Amitai Schlair
open source and project maintenance
▶ Speaking
The net BSD foundation is a 501c3 nonprofit and it's basically if you're familiar with the free BSD foundation or I'm sure open BSD has something like it. We officially accept donations that are intended to fund development in the BSD and to some extent, we determine the project direction and the structure of the organization because it is an organization, even though we're all volunteers. The disbursement of funds where we see that it helps development along.
Mike Hall
Interviewer, UGtastic
▶ Speaking
The net BSD that that's that's like the Berkeley San Diego Unix that's the foundation for the I-386 in that BSD's case. It turns out to be a lot more than that. Okay, BSD stands for Berkeley software distribution or Berkeley software development depending on when in the 90s you're looking at the expansion of the acronym. net BSD was one of the two projects that kind of forked out of the the death of the original Berkeley. The Regents of the University of California sometime in the early 90s stopped being able to fund the development that was happening in Berkeley which had been to begin with a set of patches on top of AT&T's units but eventually became its own distribution. One of the hallmarks of BSD Unix, other than actually being able to be used by humans to some extent, was that it deliberately targeted portability. It was deliberately built and distributed from multiple hardware architectures and that thereby forced to be designed to be able to work on multiple hardware architectures. NetBSD and FreeBSD are the two major projects that forked around that time out of the remains of BSD. FreeBSD chose what seems like a prescient direction at the time that they should focus on commodity hardware because mostly that's what people have. If you eke out every last bit of performance on a 90 megahertz Pentium, you'll be glad you did. Yeah, and NetBSD chose that aspect of tradition from Berkeley that let's let's turn up the knobs to 11 on portability. Let's stick with that tradition. Let's make sure we're even more portable. Let's find more ways to share driver code abstractions that let the drivers be shared abstract the way Andean-ness and other differences between platforms and just make let that drive the design which is actually why I'm here talking to you today because when I Have you heard about craftsmanship and this conference the Unix that I think goes best with that is NetBSD? I think craftsmen who use Unix and appreciate Unix and appreciate craftsmanship should be very interested in that business. Well, I think there's a lot of people here that don't realize that they're very interested in NetBSD. Didn't we we kind of tease each other about that Apple logo? But if I'm not mistaken, NetBSD is the kernel the Darwin kernel is based off of NetBSD or it's a complicated story. Naturally, the kernel in OS 10 is kind of a hybrid. A lot of its history comes from Mock, which was developed at Carnegie Mellon. Mark as the MACH, MACH, good kind of like the word Mac, but totally coincidental. It also turns out that one of the primary developers of Mock wound up being an executive at Apple later for business merger-related reasons. But the kernel is Mocked, and there is some more to it than that. That's actually not my primary area of work, but where NetBSD got involved with OS 10 is that in an early version of OS 10, I think up through 10.2 or 10.3, most of the user land utilities were from NetBSD. Early so if you for instance, if you're running user bin FTP, it's the one that that was developed in FBSD and enhanced by Luke Mewburn and distributed as an autoconf buildable piece of source. And a lot of the utilities in OS 10 were like that. Somewhere after 10.2 or 10.3, it became more of a FreeBSD influence because one of the developers from one of the founders of FreeBSD, Jordan Hubbard, was the director of Unix technologies or something like that at Apple, and so they kind of moved in a FreeBSD direction for user land, but the the kernel is actually a different story and I don't know it very well. Okay, so you know how did you come to be involved with the? You said that the kernel really isn't where you're spending your time. Where does Amitai or Schmanz, his nickname, the where Schmanz spent his time? What is your focus and what do you do with the the foundation as part of the board? I'm relatively new. I was elected some months ago and just began serving the roles that I had before that were two basically. One that I kind of self-nominated myself for like you do in a volunteer project. Whoever does the work becomes in charge, and another that I was selected for a while ago. The other one the one that I selected for is there's a project management committee for package source which is closely related to NetBSD in terms of history and organization, but isn't an operating system project. NetBSD is an operating system project. Package source is a package manager and it's distinguishing characteristics that are very compelling. What are the key features of the system you're describing?
Amitai Schlair
open source and project maintenance
▶ Speaking
The system is cross-platform, meaning it can be used on any Unix-y system, including Windows with services for Unix or SigWin, regardless of whether you have root access or not. It allows sysadmins to get the tools or services they need installed in the same way on all the machines they deal with, regardless of what they are or where they came from.
Mike Hall
Interviewer, UGtastic
▶ Speaking
Can you explain how it works in more detail?
Amitai Schlair
open source and project maintenance
▶ Speaking
It goes and grabs source code and compiles it locally, which is similar to Homebrew, a source-based package builder of packages that works on OS X. Package Source is also a source-based package manager that generates binary packages which you can then reuse and distribute. When you want to install Ruby 1.9.3 on a system, you can use Package Source to do so. The source code for Ruby is fetched, its checksum is verified, it's extracted, and if it has dependencies that need to be built and installed in the same way, they are installed. One of the special things Package Source does is to make sure that the build environment on the user's machine is identical to that on a development machine, so the package will be reproducibly buildable. At the end, you have a custom-built Ruby that's only for packages from Package Source, and it's put in your path.
Mike Hall
Interviewer, UGtastic
▶ Speaking
Does it also resolve dependencies and things like that?
Amitai Schlair
open source and project maintenance
▶ Speaking
Yes, it goes all the way down and resolves dependencies. When you're talking about Ruby, there are various packages Ruby management tools like RVM. RVM needs support, and I'm not a Rubyist, but I understand that it's an excellent question.
Mike Hall
Interviewer, UGtastic
▶ Speaking
Are you familiar with an equivalent problem with Pearl where you want to have different versions of Pearl with different gems in the Pro world installed, and be able to switch between them and know which one you're currently working with?
Amitai Schlair
open source and project maintenance
▶ Speaking
Yes, Pearl has ways of doing that. Package source is kind of orthogonal but can be used for that specifically. What I mean is that you can take one package source source tree and bootstrap the package source tools and an entire set of packages as long as it's in a different location, it's totally independent and the parameters can be different and the values can be different. So I've used that in the past to get one set of packages look like this, another set of packages in a slightly different place look like that, and you can move a sim link or do some other trick right.
Mike Hall
Interviewer, UGtastic
▶ Speaking
How did you come to own the work and volunteer for it? How did you end up even getting into position to volunteer for that one?
Amitai Schlair
open source and project maintenance
▶ Speaking
NetBSD's website unfortunately doesn't sing the virtues of the operating system in the package manager as well as it should. The virtues are terrific, but the website is not so so. I took a hard look a few years ago at why we're having this problem, and some people think it's because developers don't like to write documentation. I don't believe that's true, especially among craftsmen which most of the NetBSD developers would agree that they are. I think the problem is one of tools that we built ourselves that get in the way. We built a while ago based on old ideas. I think what all of us would agree now is that the reason of the way to do web content in this day and age is something more like a wiki with a simple input format, simple output format ideally you can edit it with VI or Emacs in addition to a browser.
Mike Hall
Interviewer, UGtastic
▶ Speaking
What piece of software did you find that was close to what you needed?
Amitai Schlair
open source and project maintenance
▶ Speaking
I found a piece of software that was close to what we needed. We had a bunch of tight requirements to be able to run the software. I'm also a member of the sysadmin for the NetBSD project, so I knew enough about the web problem, I knew enough about what the administrators would be happy integrating, and I knew enough about this piece of software that would bridge the gap. That's why I'm also a contributor to wiki, which is an open source written in Pearl content management system. So I extended wiki so that it met NetBSD's requirements. We stood up an example of it, the admins liked it, some users
Mike Hall
Interviewer, UGtastic
▶ Speaking
liked that we now have wiki.netbsd.org and someday when I have time that will be how our website is made okay so I just kind of volunteered for that and now I'm on the web committee even though I didn't really mean for that to happen yeah but it's an okay way yeah that it sounds like it's a lot of fun and but you know even going back further why NetBSD what was what attracted you to NetBSD to begin with great question so I went to high school about 20-25 miles north of the road from here and in high school the available nerd tooling that I could get at was Texas Instruments Graphic Calculators at the time that's all the programmability that I could get to it so I got to it I had one of everything I had 81 82 85 and 92 which was two megahertz faster than the CPU in the Mac plus we had at home oh wow so it's just like today when you have that that little iPhone and a three-year-old computer it's like no they're shameful yeah but so that's what I had and and I got into the community around it and was invited to participate in a community project that we were we were building called ti calc. org ti themselves wasn't real forthcoming supporting all the third-party things that were happening so we wanted to be for ourselves and I was invited to be part of that and they were running on some weird system called Linux one point something I don't know the first thing about it right so I said how can I even be useful to you guys if what I have is you know an old Macintosh Mac 2ci at the time how can I even learn how this stuff works so that eventually I can be useful to you guys and one of the people on the project said oh if the Mac 2ci is what you have then you should run that BSD so the 2ci that's that's pretty that's pretty ancient that's so you dated yourself there I didn't even realize that I mean pre i386 that you know pre Intel processors sure that you could run unix on an old Apple that far so I mean that the support goes back that far it does net BSD runs on old 68k Macintoshes Amiga's with the similar chipset weird old machines I don't even know what they are a limited run set of machines called the shark that had I think an arm based processor but not like the ones we have in our phone yeah yeah and that was famous for being silent and having no fan it's also famous for having almost no performance but that BSD runs on it pretty much anything with an MMU is enough and somebody will report to it if they have an interest and by the same token that BSD NetBSD has a reputation for prioritizing compatibility with old machines even when it's impractical and gets in the way. Is this true?
Amitai Schlair
open source and project maintenance
▶ Speaking
Yes, that's correct. NetBSD is known for maintaining compatibility with older hardware, which can be beneficial for those who need to run legacy systems. However, this compatibility comes with a maintenance cost for people who are doing kernel and system development, as they have to deal with these older systems. The cost is not exorbitant, and there's no reason to be overly concerned about it. NetBSD also runs well on modern hardware, so it's a versatile choice for a wide range of users.
Mike Hall
Interviewer, UGtastic
▶ Speaking
If someone is looking to run NetBSD on a Mac and wants to explore its heritage, is it possible to run it in a virtual machine?
Amitai Schlair
open source and project maintenance
▶ Speaking
Yes, it is possible to run NetBSD in a virtual machine. NetBSD has been part of the early development of the XEN system and has excellent support for XEN both as a DOM 0 and a DOM U. You can download NetBSD as an ISO or tarball, and there are systems available for free download. Additionally, there's a free shell account available for experimentation. To get started, you could use package source, which is available on various platforms, including Mac. I was one of the people involved in porting package source to OS X back in 2001/2002, making it one of the first platforms that wasn't NetBSD to receive this feature. For me personally, when OS X came out, I used to have to have two different computers: one for my Mac and another for NetBSD. The Mac would crash at the slightest provocation, but I could at least SSH into my Mac 2 CI and then NetBSD over there. With OS X, I could see a possible future where I only need one computer, and package source bridges the gap between whatever Apple puts in or leaves out, allowing you to catch up to the exact same versions of things that I would have on my NetBSD system.
Mike Hall
Interviewer, UGtastic
▶ Speaking
Do you recommend people who are interested in learning about NetBSD's heritage and running it on a Mac to take a look at package source?
Amitai Schlair
open source and project maintenance
▶ Speaking
Yes, I would recommend it. Package source is a powerful tool that allows you to manage and install software packages on various platforms, including Mac. It's a great way to stay up-to-date with the latest software and libraries, and it's compatible with NetBSD. By using package source, you can easily catch up to the exact same versions of things that you would have on your NetBSD system, making it a valuable resource for anyone interested in exploring NetBSD's heritage and running it on a Mac.
Mike Hall
Interviewer, UGtastic
▶ Speaking
using homebrew and maybe reconsider homebrew or? Well I'm speaking from ignorance because I haven't used homebrew more than a time. I have a good clone of it just so I can see if they have a recipe that I want to borrow something from because you know they they're focused on OS X so they may have something that package source doesn't have yet and wants to have but I've actually run it I have used Mac ports once or twice a long time ago I tried Fink I have checkouts of them as well for the same reason but I actually don't know enough about other systems because I've been so happy with package source for so long because I can do the same thing and all the machines that I have so I couldn't even make an informed opinion about that.
Amitai Schlair
open source and project maintenance
▶ Speaking
So actually a question about package source because I alluded to is it also a little bit like Chef it sounds like you can also script a system to build? You can so there's a there's a concept of bulk builds which obviously we do in the general complete case on auto build systems to see what's broken and what we've changed and to generate binary packages but a person can also drive their own private bulk build in whole or in part if they have a specific set of packages they want and wind up at the end binaries they can just sub in for the ones that they have so yes so and so if I wanted to say oh I have a brand new install of a NetBase VSD server or maybe even a Linux server and I want to just say go and put all the you know put in MySQL put in Postgres whatever I can use one of those bulk build scripts to just go off and build my system and then use that to reproduce across different servers exactly that that also includes things like compile-time options like if you you have MySQL and you want to build PHP with support for it or you want you know you're you're setting up dovecott and postfix for your mail server and you want them to be able to store a gray listing database in MySQL or in Postgres it's a compile-time option you define that in a configuration file and the package build finds it so the bulk build also finds it and what you wind up with at the end includes the choices that you made and with doing things like like if I script to build for a new server I'll have something that goes and goes and fetches the dependencies builds the the piece and then a separate or builds the thing and then maybe a separate shell script that I call after to maybe do some post install configuration can you hook into any of that stuff with package I'm sure you can in fact I
Mike Hall
Interviewer, UGtastic
▶ Speaking
went to package source con in Berlin in March and one of the best talks I saw there was about Ansible which is one of those okay promote system and driving systems and this is a person who's been a package source developer I'm sure I know Ansible is a system of shell scripts so you can easily hook in a call to package add and God knows what else I imagine chef and puppet and those guys could do the same well it's very interesting because when you mentioned Ansible Ansible is one of those things that I've seen on the periphery but never was like oh that's just something that somebody's out there but then when you talk about this shared heritage now that it has this relationship with package source it's now much more compelling for me to go and spend time it's not an official relationship but it seems like it's a natural fit okay so Ansible is a thing to check out if you're also looking at package source um if if so now I just I just messed myself up now because I lost my flow because I took a took that aside to point at the camera um so you're here at SCNA and you're interested in in the craftsmanship uh conversation and you said that it seems like a natural fit for what the core of NetBSD is but if somebody's here at SCNA and they're they're they're looking at how I can contribute back to open source um is is is the NetBSD something you might point them at saying go go look at bugs or where would they go look to you I would absolutely recommend that BSD I think it represents uh if if what people here are interested in is operating system development NetBSD presents a unique value proposition compared to any other open source system they could volunteer for specifically first of all as a craftsman you want to have whatever you're building on to have been made in a craftsman-like manner which I think one of our speakers yesterday uh alluded to that you you want the tool that you were given to be made with quality and then you want to use it to pay it forward to build something else with color NetBSD for whatever you might want to build on top of it uh is a very simply designed coherent no funny surprises piece of software which for an operating system is a serious achievement right and then as a craftsman in the world of business you want to have the option to extend software without necessarily having to publish the changes that you make and that BSD license-wise like any of the BSDs is superior to Linux in that regard you have the option of making changes that you can keep private for as long as you want to um what's the BSD license the BSD license as long as you give
Amitai Schlair
open source and project maintenance
▶ Speaking
the BSD license is a permissive free software license with conditions that ensure the software remains free and open-source. It allows users to use, modify, and distribute the software, as long as they provide attribution and include the original license and copyright notice. The BSD license is known for its simplicity and flexibility, making it a popular choice for many open-source projects.
Mike Hall
Interviewer, UGtastic
▶ Speaking
What sets NetBSD apart from other BSDs?
Amitai Schlair
open source and project maintenance
▶ Speaking
NetBSD differentiates itself by defaulting to every build being a cross build. This means that the toolchain that builds the system first bootstraps a toolchain from whatever your host system is to whatever the target system you want to build for. This allows you to use whatever your fastest computer is to develop for whatever system you're targeting, even if it's an embedded device with a totally different architecture. This is baked into how the system gets built all the time.
Mike Hall
Interviewer, UGtastic
▶ Speaking
Can you explain how NetBSD's approach to testing is unique?
Amitai Schlair
open source and project maintenance
▶ Speaking
NetBSD has a unique culture of testing that is not typically possible in a kernel. They have special technology in the kernel called 'Any Kernel,' which allows any component of the kernel to be targeted either for direct linking into a monolithic kernel or compiling it into a thin layer on top of some other system or into a standalone program with all the source code in it. This means you can write automated tests for kernel code that, when they break, don't mean your system crashed; it just means your process crashed, and your test harness can keep running, record the failure, and provide fast feedback. This is not only about security but also about testability.
Mike Hall
Interviewer, UGtastic
▶ Speaking
How does this approach to testing benefit NetBSD users?
Amitai Schlair
open source and project maintenance
▶ Speaking
This approach to testing benefits NetBSD users by providing fast feedback and the ability to identify and fix issues more quickly. It allows for automated testing of kernel code, which can help catch bugs before they become critical. This recognition that simplicity and orthogonality go hand in hand is a key design principle in NetBSD, as it leads to more powerful and flexible systems.
Mike Hall
Interviewer, UGtastic
▶ Speaking
all these options of how to exercise it yeah well uh thank you very much for taking the time to sit down here really appreciate it and uh thank you
Amitai Schlair
open source and project maintenance
▶ Speaking
and then net BSD Foundation is something you should check out what was the uh URL www. netbsd. org netbsd. org thank you very much thank you
Mike Hall
Interviewer, UGtastic
▶ Speaking
[Music]