Memories of a very dear friend

It's been a week since we got the sad news that a dear friend of ours had passed away unexpectedly.  Since that time, David has been in my thoughts a lot.  Because I had the pleasure of working with David at 5 different companies for a total of 16 years and had also enjoyed his company after work on a regular basis, I know a number of his work friends.  I tried to help share the news of David's passing.  One of our friends responded by commenting how well David and I had clicked which made me smile.

I met David in 1990 when I left a contracting position at the Washington Navy Yard for a job with Sprint International which created equipment for Sprint's packet switching network.  My boss at Sprint was a Brit I had previously worked with at GEISCO.  My boss was a firm believer in getting out of the office at lunch and arranged regular outings to various ethnic restaurants in the area, something he had also previously done at GE.  I think it was at one of those lunches where I first met David.  As we chatted, it became obvious that we had similar tastes in books and movies.  We started eating lunch together on a regular basis.  My boss was also keen on regular after work happy hours, which occasionally involved playing darts.  David and I enjoyed many of those gatherings, sometimes playing darts.

Initially at Sprint, David and I were working on different types of PADs (packet assembler/disassembler) network nodes at Sprint.  I was working on a QLLC PAD which enabled IBM equipment to connect over Sprint's X.25 network.  David worked on more traditional PADs until Sprint started developing Frame Relay, which was a hot new network technology at the time.  Frame Relay was the perfect place for David as he picked up new technologies so easily.

After a couple years, I transferred to Sprint's Operating System group which gave David and I more chances to work together.  I enjoyed that opportunity since I discovered that he and I had compatible troubleshooting approaches.  I believe that came from both of us having started out working on computer hardware, him designing it and me repairing it.  

David and I enjoyed getting together outside of work which continued after I left Sprint in 1994.  It was at one of the after work gatherings that I remember us arguing over the actor's name who had uttered a line in the movie "Cool Hand Luke".  Fortunately, there was a movie store next to the restaurant where we were.  David and I left happy hour for a few minutes so we could consult the movie guides next door to settle the argument.  Other people at happy hour laughed at us for needing to prove our geek cred that way.  That's just one of many little happy moments with David that make me smile when I remember them.

The next time we worked together was in 2000.  At the time, David was working for a startup company which was developed financial problems, as many small startup companies do.  I was happy to help him get a job at 3Com where I was working at the time.  I was developing firmware for a number of ADSL modems while David was helping the ADSL architecture group.  This was another ideal position for David since network architecture requires expertise in so many different areas and David was always eager for opportunities to learn new technologies.  3Com was challenging since the schedules were incredibly aggressive.  That was because the group we were part of sold hardware to consumers which is a rapidly changing environment.

Later in 2000, David and I were both contacted by someone we both worked with back at Sprint who had taken a management position at a new optical networking startup company called Ocular Networks.  We both took the plunge and joined within a month of each other.  Ocular gave us the chance to work closely together on a regular basis.  David initially worked on a DS1 board while I worked on a DS3 board.  These boards provided electrical network interfaces which could be concentrated over the fiber optic network cables.  Like many early stage startups, Ocular required that we work 60-70 hour weeks for the first couple of years.  Fortunately things slowed down a bit after Ocular was purchased by Tellabs.  Tellabs moved our office from Reston to Ashburn.  It wasn't long until we discovered that our new office was very close to the Old Dominion brewpub which became a favorite place for after work gatherings.

The group of engineers I had met at Ocular, many of whom I had worked with at other companies, were so nice that shortly before I left Ocular in 2004, I organized an e-mail list to make it easier to organize regular get togethers as people left for new companies.  I often think how ironic it is that someone as naturally introverted as I am ended up in the role of organizer for activities outside of work.  That's due in large part to David and a few others like him who I couldn't bear the thought of losing contact with.

After Ocular, I took a position at AOL in the e-mail server group.  David also moved to AOL shortly after that and we found a way to work together again.  Unfortunately, the group we were with disbanded before too long and despite being moved to another group together, we both ended up leaving AOL.

For the next 10 years or so, we only saw each other for lunch or for an after work happy hour but fortunately those meetings were regular enough so we didn't lose contact.

In 2015, David joined a company called FourthWall Media, where I had been working for 5 years.  We got to spend another 5 years working together before a shift in company direction resulted in both of us getting laid off.  FourthWall was fond of company outings and I have happy memories of baseball games and visits to Top Golf to unwind.

The pandemic limited our in-person get togethers for a while but since I hated the thought of losing touch with David and a few others, I started a weekly video chat call on Skype which has been something we all looked forward to each week.

Over the 35 years I've known him, David has been someone whose company I have enjoyed and whose opinions I have valued.  He was an absolute joy to work with since he's very knowledgeable and extremely easy to work with.  We've shared recommendations for books, movies, music, and beer.  In the week since I learned of his passing, I've encountered a number of things which I wanted to share with him only to remember that he's no longer available.  He will be missed more than words can express.

Here's a picture of David and me at the Lost Rhino Oktoberfest in September, 2023.  I'm at the front left and David was directly across from me, looking at his phone.  This photo makes me smile because we spent part of the day trying to answer trivia questions posed by the musician who was playing.  We were up to our old geeky tricks that day.

Configuring Windows/Mac/Linux for embedded development

A few days ago Scott Hanselman posted an interesting question on Bluesky.  He asked how much stuff people needed to add to Windows to make it useful for day to day work.  He also asked a similar question of Mac users.

Admittedly, my use case differs from that of most people.  I do embedded firmware development.  For me, my company Windows laptop mostly acts as a way to connect with the Linux build machines and target machines I use.  It's really little more than a glorified terminal except for running Outlook, Office, and Slack.

Windows

Having made the switch to a Mac at home 24 years ago, I only use Windows at work now.  On any new Windows machine, I first install the following software.  It's all free software as most companies I've worked for make it so difficult to justify the purchase of commercial software, that it's not worth the effort.

• Gvim - I occasionally need to do some local editing on Windows and for that a graphical version of vi is an absolute necessity for me.  I've been using some version of vi for 35+ years and while I've had occasionally dalliances with other programming editors, I've always returned to vi.
• VcXsrv - Being able to launch graphical applications remotely makes my life much easier.  That means using an X11 server.  I know there's pressure to move to Wayland but it strikes me as more effort than it's worth at this point.  It's the same feeling I have when I hear someone suggest that I try writing a device driver in Rust.  I just want to get work done, not spend time blazing a trail.
• Putty - I need to connect via SSH or serial communications to a number of Linux machines (build servers, target systems, etc) and Putty is my hands down favorite way of accomplishing this.  I make sure to enable X11 forwarding on Putty SSH sessions because this allows me to launch GUI programs and have them display on my Windows laptop.
• WinSCP - This allows me to easily copy files back and forth between Linux machines and my Windows laptop.  It also enables easy remote editing of files which reduces the pain of editing a file on a remote machine over a slow Internet link.

Mac

When I first started using a Mac at home, I loved the development environment which the combination of Mac OS X, Xcode, and the Quartz X11 server provided.  It was the best development platform I had seen since my days last using a Sun workstation in 1996.  Over time and Apple's push to combine features of iOS and Mac OS, it's become much harder for me to set up a reasonable development environment on the Intel Mac Mini which serves as my desktop machine at home these days.  Since most of my embedded development is done for work, that's not a deal breaker.

• MacVim - As mentioned above in the Gvim section, I need to edit files locally on my Mac.  MacVim gives me a version tailored for use on Macs.
• Homebrew - Unfortunately, many of the tools I've come to rely upon are only available through an alternate install path.  Homebrew gives me access to a number of development tools not available through the Mac AppStore.
• XQuartz - This X11 server used to be available in the Xcode tools but now the best version seems to require being installed via Homebrew.
• Unfortunately I have not found a free GUI SCP application for Mac I like yet so I resort to using the standard Mac Terminal app and the command line scp tool.

 Linux

I use a Raspberry Pi 5 at home since Linux is orders of magnitude better at interfacing with a variety of small embedded machines than either Windows or Mac are.  I typically use a pared down Linux distribution because I don't need the typical blend of applications like Open Office.  I've been using Debian Bookwork with the Xfce desktop environment.  

It's easy to install X11 apps, Gvim, and Putty on Linux.  The IT group at work has our Windows laptops very locked down so installing new software such as the GUI software for a USB protocol analyzer sometimes requires getting it approved which can take a few days.  Mac has gotten harder to run third party application software as well, much like the iOS app store which is very locked down.  Development goes so much faster when I can install any software I need without facing roadblocks.

Linux is also good at doing compiles for the firmware and application software I create for the newest embedded ARM device at work which is also an ARM 64-bit processor.  It has better USB support too.  Windows often requires the installation of device drivers for various USB serial devices which can be hard to do when using a laptop with limited admin rights.

Experience versus enthusiasm

We live in a somewhat rural area which means we have a well as there's no municipal water supply available.  Last week we discovered that we had no water pressure.  Since our house is 24 years old and supposedly well pumps seem to have an average lifetime about 20 years, this was an inconvenience but not a huge surprise.

What I found interesting was observing what it took to get the problem resolved.  The plumbing company we called did an excellent job.  They had a young plumber out to diagnose our problem within 4 hours of us reporting the problem.  The plumber they sent was very nice and extremely diligent.  Since he dealt mostly with houses on the eastern and more suburban portion of our county, he wasn't familiar with wells.  However he was able to get advice on how to troubleshoot the problem from more experienced plumbers at his company and after 3 hours, he determined that our well pump had finally died. 

The next day he returned early with a more experienced plumber (one closer to my age) who was familiar with wells and rural water supply equipment.  The two of them worked hard to replace our well pump in very cold temperatures (15-20°F).  During the times they came into the house, I had a few chances to chat with the more experienced plumber and found him to be not only very knowledgeable but also a really nice guy.

It struck me after they had left that the older plumber and I have found ourselves in somewhat similar situations.  I'm one of the two oldest engineers on our team at work and I'm definitely the oldest who still works full time.  I work on things that the younger engineers don't have experience with such as firmware, device drivers, and operating systems.  From time to time, the need to deal with old technology such as a serial port crops up and I'm happy to do it because it brings back memories of a simpler time.  I also seem to get all the core dumps to analyze which I find to be challenging puzzles.  Who needs brain teasers like Wordle when I can spend hours solving a crash?

I guess the lesson to be learned it that it's useful to have engineers of varying degrees of experience on a team as learning from people who have been around some type technology longer is more efficient than younger techs having to learn everything on their own.

WordStar

I recently commented on a post on Mastodon about Wordstar by Tom Jennings (yes, the one associated with FidoNet).  In his post Tom extolled the virtues of Wordstar for what a good piece of software it was and I completely agree.  Not only was it good for its time but it compares quite favorably against modern software.  It needed to be configurable because software which ran on CP/M often required customization of the display and printer settings to match the hardware connected to the user's machine.  It was also quite robust.  I last used it around 1989 and I don't recall it ever crashing.  I cannot say the same about any modern word processors I use.  Finally, it was remarkably full featured for its time.  I recall being excited to discover that it had a column editing mode which at the time I had only seen on IBM's PE (Personal Editor).

I appreciate both Mastodon and Bluesky because they allow me to see what favorite authors, scientists, engineers, and artists are up to at the moment. 

 

A few of my favorite programming environments

In my 40+ year career in software/firmware engineering, only 3 of the computers I've used have had what I considered superior development environments.  This type of environment makes it easy to be more productive.

The first was the Sun 3/80 Workstation i used at work in 1990 while writing software for packet switching communications equipment.  It had a Motorola 68030 CPU which is still my favorite assembly language and offered good performance.  The Sun offered my first extended exposure to Unix and it was love at first sight.  Unix was mature by that time and the development tools were top notch.  The only possible drawback was it only had a monochrome display albeit a high resolution one.  I enjoyed the way it allowed me to define a default window layout so I could have 4 terminal windows in standard locations each time I logged on.

The second machine I thought offered a great development experience was the Apple PowerMac G4 I bought for home use in 2002.  It was purchased to replace a PC because I had grown tired of Windows XP plus I had picked up an iPod 10 GB music player which integrated much better with Mac machines than it did with Windows.  The PowerMac G4 ran OS X, a Unix like OS derived from NeXT's NeXTSTEP. It had a PowerPC G4 CPU which was my first exposure to the RISC architecture.  OS X came with the X11 (aka X Windows) window manager which was also used on the Sun workstation.  X11 enabled me to run many of the same tools I had grown used to using on the Sun.  At the time I was working for an optical networking startup.  Some of the boards in our networking equipment used PowerPC G3 and G4 CPUs so it was handy having one at home to experiment with.  The Mac also gave me an environment which was very close the the Sun I had started using 12 years earlier with the added benefit of supporting a color display.  I've come to love color syntax highlighting of source files.  I eventually made the switch to Intel Macs and then Apple's own CPUs.  Sadly, they also seem to be making Mac OS look more like the iOS used on their mobile devices at the cost of making it less useful as a development machine.

The third (and my current favorite) development environment I've considered exceptional is a Raspberry Pi 5 which runs a multicore 64-bit ARM CPU.  It still amazes me that I could built a reasonable development machine (8 GB memory, 500 GB SSD storage) for less than $200.  Since it runs Linux with X11, I'm once again able to run the same tools (or their successors) on this machine.  It integrates better with the small embedded ARM machines I write firmware for than does my work laptop running Windows 10 or my Mac Mini which runs the latest iteration of MacOS.  Sadly X11 is no longer easy to get running on the Mac.  I know some will think I need to migrate to Wayland but I rarely feel the need to use tools which aren't fully mature yet.

All of these machines have run Unix or Unix-derived OSes which has been my strong preference for 35 years now.  This allows me to edit source files using vi (now vim) which had a steep learning curve but rewards the initial effort by being fast and quite powerful.  Having ctags allows me to quickly jump from a reference in a source file to where the function or variable is defined.  I've used it for so long that it has become second nature to me.  I'm happy to be able to search my source code efficiently again using grep.  And I love being able to use a shell with a proper scripting language (currently bash but I've used sh, csh, and ksh in the past).  Best of all, I've still got good support for the C programming language which I view as a portable assembly language and which has been my favorite since I first learned it in 1985.

These aren't the only environments which can improve productivity.  I've worked with a few engineers who swear by Visual Studio or Eclipse and I've seen how both can help speed up common tasks.  However both have a sufficiently steep learning curve for me to ignore them.  I feel the same way about Emacs when friends point out its advantages over vim.  At this stage in my career, I'm not willing to make radical changes to a workflow I've been comfortable with for a long time at the cost of weeks to months of feeling less productive.

Antisocial with a reason

I'm sure people have noticed that I've been much less social since the start of the pandemic.  That's in large part due to the fact that my wife has autoimmune issues which put her at increased risk from viruses such as Covid-19.  To limit the likelihood of her getting infected, we both mask up whenever we're inside in public spaces.  While this is acceptable for quick shopping trips, it doesn't work well for meeting friends in restaurants or at parties unless there's an option for outdoor dining.

The need for outdoor dining has limited when it's feasible to meet people to times of the year which offer reasonable temperatures to brave the patio.  Those times happen to coincide with when I've got 5+ hours of weekly yard work to complete.  The need for dry weather while mowing can make it difficult to find a mutually agreeable time to meet.  Sadly, patios tend to fill up during prime dining hours.

It may come as a surprise to people that some of us need to be careful because the media offers so little coverage of Covid-19 these days.  The common wisdom seems to be that Covid-19 is no more dangerous than the flu which is definitely not true.  It's easy with an Internet search to find wastewater data which helps track infections through monitoring of sewage treatment (see link below).  Sadly this doesn't always paint a completely accurate picture of infection levels in more rural areas such as ours where municipal water and sewer service are not available so we're forced to infer using the data from surrounding areas.

https://www.cdc.gov/nwss/rv/COVID19-currentlevels.html

My wife and I both miss seeing friends as often.  We also miss live music and dining out.  Hopefully we'll be able to eventually return to doing the things we enjoy without the need to be so cautious.

Sometimes it pays to be skeptical

I may have been born a skeptic.  I've been questioning things I was told for as long as I can remember.  I'm sure many of my teachers were happy to see me advance out of their classroom because of that.  In many situations that doesn't make you popular, however it can serve you well in an engineering career.

On occasion I've needed to be skeptical of things colleagues tell me.  Such misinformation was most prevalent when I was a field engineer (aka FE) 40+ years ago.  If you're not familiar with that title, it's basically a mechanic for computers.  In my first job in the computer industry, I worked on mainframes and minicomputers.  For part of that time I was a specialist which meant I got called in on difficult problems after other engineers had tried and failed to fix.  I started these visits by asking questions of the FEs onsite only to sometimes have them tell me that of course they had checked the things I was asking about.  I learned which engineers I could trust to admit they hadn't checked something which seemed a logical troubleshooting step.  The challenge with engineers I didn't know well or with those I knew were too proud to admit they had missed something was to suggest that we check something together which they had assured me they had done already without embarrassing them too much.

These days my skepticism allows me to discover the discrepancies inherent in technical documentation.  I don't recall ever seeing a chip datasheet which didn't have a few errors (or instances of wishful thinking on the part of the documentation team).  Accepting the idea that the documentation can be wrong allows one to move beyond seemingly impossible situations such as a device register which occasionally isn't as persistent as the manufacturer's docs suggest.  Software documentation is frequently more error prone than hardware documentation.  I don't think I've ever seen an API document without a few mistakes.

Comments in code is another area it's dangerous to trust blindly.  Engineers will often add extensive comments in code when a function is first created.  Subsequent revisions may not see those comments updated to reflect changes in logic.

That makes the world of engineering seem somewhat bleak.  How do we combat it?  For my part, I try to report errors I discover.  That doesn't always work.  I've reported errors in compilers my company has had to pay healthy amounts of money to license only to be told that the compiler is EOF (end of life) and that no errors would be addressed.  I couldn't even convince the vendor to add my discovery to the list of known bugs.  The thing which keeps me trying is occasionally someone at a vendor will be appreciative of having a bug reported.