The 8 Year Rule

In a recent kerfuffle, Sonos (makers of mid-high speakers) announced that out of necessity, they’d soon cease support for new updates to some of their older equipment. Twitter got mad, bloggers blogged, tech pundits bristled while others apologized. It was a whole thing. The Sonos CEO issued further statements to add clarity and to try to calm the mob.

In the hobby of retro gaming we are constantly reviving 30+ year old gadgets. It occurred me that in other parts of my life (when looking at products) I operate under a proclivity to be selective in some cases and lax in others in adapting new tech. As as much an exercise in introspection as explanation, I thought it might be good to look at this topic and ultimately consider how to might apply to the retro-gaming, arcade and pinball hobbies.

A while ago, I got a text from a pinball-player friend.

totangrabhr.png

I met him in front of a 80’s constructed home on an oversized lot in one of the largest subdivisions in our region, about 3,000 homes. He was playing coy b/c he had found a steal. As we walked down the basement to find his snag, I felt like we were transported to 1996. 1990’s furniture (in surprisingly good shape) buffeted a room with 1990’s Bose sound equipment, a ping pong table, a period entertainment center and in the corner of the room a mid-90’s Williams Pinball of some renown: Tales of the Arabian Nights. 25+ year old electronics, all of it.

The thing is, the stereo and tv still worked. The pinball machine needed some repair (which was why he bought it as a fixer-upper) but it was totally repairable with skilled hands and bench-typical soldering tools.

A Thought Experiment

Now, imagine the technology you have today in your living room, your gameroom or your pocket and imagine jumping ahead 25 years to walk in the room exactly as it is today. Sort of like Sam Flynn flipping on the lights in his Dad’s arcade.

How well do you think your smartphone will work? If the battery hasn’t exploded in the intervening time, the cellular network will surely no longer support the same mixture of communication protocols and radio frequencies we call LTE. Probably a commercial running on the holo-emitter for “17G, from Disney Wireless: All the power to drive your 20k holo-displays.”

Your smart-home devices, will they work? “Alexa, turn on the pinball machines.”
The internet will look dramatically different 25 years. More than likely having long sense dropped support for IPv4. Your cable or DSL modem will no longer have a compatible peer to connect to. No way 2025 is still using DOCSIS 3.1 over hybrid-fiber-coax. DSL probably won’t even exist.

The fancy fridge you bought with a giant iPad on the door - still doesn’t make ice correctly and while it keeps your food moderately cold, the display’s internals have long-since failed. The wifi-connected stove doesn’t turn on anymore. The motherboard has failed in your smartphone-accessible Washer and Dryer.

I know, it’s a long walk to a known conclusion

Technology ages in dog-years.

Specialized & physical vs generalized & virtual

That 25 year old Pinball machine is powered by solid state through-hole components. Electrolytic capacitors, carbon composite resistors, field effect transistors. The smallest components are still easily serviceable with a desoldering pump and soldering iron in most cases. Even better, the full original board schematics are available for free and are part of the public domain.

Simple logic gates and good electrical design make that 25 year old boardset as easily repairable today as it was 25 years ago. What’s more, it is repairable with 40-50 year old tools. I can fix nearly anything on these games with a good multimeter, diode tester, soldering tools & a logic probe. Hell, even the sound and logic ROMS are available so you can burn new ones.

WPC-95, WPC-DCS, SAM, Sega WhiteStar, DE V3 are all pretty simple to maintain with common bench tools.

No operating system, just (relatively) simple, well-documented logic circuits.

But, technology must advance. Excelsior!

Most modern JJP, Stern or Chicago gaming machines have some flavor of Linux at the bottom of the stack. For you non-PC folks, Linux is an Operating System, like Windows but not. MacOSX under Intel architecture is a Linux-adjacent operating system in some ways. Embedded Linux and Embedded Windows are what we call it when the OS runs as part of a distinct hardware framework or chipset. Odds are that your media center DVR, your smart fridge, your connected-washer & dryer, your smartphone, your Alexa Speaker and yes: even your Sonos speakers are running some flavor of an embedded Linux OS.

Operating systems are bootstrapped hardware abstraction platforms. Layers of hardware-interface software (drivers) with numerous common services and platforms on top. These services encapsulate and simplify more complicated tasks. Things like the 3d graphics and physics engines OpenGL or DirectX on PCs or METAL on iOS. Things like a Network IO stack for communicating with other machines and local IO Services for interfacing with peripherals. When you have to wait 6 minutes for Windows to boot or 45 seconds for your old Android phone to boot - you are watching the hardware load the operating system and related services.

SimpleStack.jpg

Embedded-circuits are designed and purpose-built to provide a certain set of behaviors time after time. There is no operating system in the traditional sense outside of instructions encoded to roms and processors. A real “computer” has an operating system and is suited for tasks that require higher complexity or more dynamic. Ultimately, you can sometimes get away with using a computer beyond the limited constraints of its limited design but an embedded circuit will rarely show the same level of adaptability.

Stern_1980_Quicksilver_MPU_Schematic.jpg
Stern_1980_Quicksilver_MPU_Schematic2.jpg

Without knowing the symbols the pinball diagram looks incredibly complex and difficult but if you trace functions by pinout to their related circuit you can use these schematics with success to get to your problem areas. The key is reducing your focus to parts of the board related to specific feature features. In the schematic above, when a switch that is ultimately connected to J2 on the board no longer works, you can pretty readily isolate all of the components in the circuit to find your problem area.

Software stacks are often more complicated to troubleshoot, sometimes by design. The schematics for your 25 year old pinball are available but the source code for Windows (usually) isn’t. In addition you still have the risk of underlying hardware that will eventually fail but with often less-clear symptoms.

Through-Hole vs Surface Mount

Modern logic boards (PCBs, motherboards, etc) are comprised of mostly surface mounted components while older equipment was often through-hole. That is to say that an older Arcade or Pinball mainboard might have a bank of dozens of resistors and half-dozen transistors that are stuck through holes in the board and soldered in place, interfacing with traces that were printed on the board. By comparison, modern PCBs are mass-produced and often the components are laid down in a very specific grid (envision the mesh that holds together tile backsplash but super-tiny) and later applied to the host board with hot-air.

Surface-mount components are a fraction of the size of their through-hole counterparts, in most cases. They are also often “different” in some way, internally. For instance, 80’s and 90’s resistors are usually of carbon composite construction where modern resistors are “film” constructed. Guess which ones are more reliable? It’s debatable. In theory, a smaller component with less conductive material should generate less heat. (Heat is the enemy because temperature cycling contributes to component failure over time.) But, smaller isn’t always cooler. When smallness is achieved be densely packing components together the heat scale can often go the other way.

A modern i5, i7, i9 processor bottoms out at around .014 micron. That’s equivalent to 2.7 billion transistors packed into a roughly 1 inch square. It gets hot.. quickly! Hot components = reduced service life.

Side Note: If we were to print a schematic for the typical i7 quad-core processor with the same scale as the pinball PCB schematic above, it would wrap around the earth at the equator once. So that other schematic seems more manageable eh? :)

A Tech stack, for Tech stack’s sake

Modern Pinballs, Arcades, Refrigerators, Smart Speakers, Dishwashers, Toasters and more have tiny low-powered computers on them running a manufacturer variant of some operation system or another. This allows your smart speaker to connect to your favorite music streaming service over wifi, your Sonos multi-room audio to sync up, your Galaga/MsPac Reunion to boot an emulated version of the original games.

We’ve been in a trend where “tech layers” get added to everything from kitchen appliances to the company operations of luggage manufacturers and other traditional companies.

Hey, computers are great and tech can enhance our lives. When you attach a computer to an appliance you immediately speed up the aging process for that appliance. Today’s smart fridge with 3 cameras and a smart assistant is tomorrow’s bricked android device.

It wasn’t uncommon for our parents to get 30 years out of a dishwasher or washing machine. Now, they have motherboards (single-board-computers or SBCs) that run a software stack to enable the smart features. Even your LG, Samsung, Vizio Smart TV is partially monetized from aggregate data collection on how you watch it and which smart apps you choose. You totally read the privacy policy, right?

This marriage of technology provides a wonder of convenience, sure. But will that LG wifi-enabled Dishwasher stand the test of time? It isn’t reasonable to assume so. I’m a programmer. I’m proficient in many languages, can muddle my way through anything from an a mainframe to cloud services backed machine learning running on a smart phone. Even I.. a card-carrying nerd find myself eyeballing needless tech in appliances as potential future weaknesses.

 

8 Years

The number I’ve settled on is: 8 years. That $1000 wifi-connected washing machine? In 8 years you’ll (probably) need another. That Nest Thermostat? Plan to replace it in 8 years. The crazy-awesome technology features in your new Ford? 8 years.

Granted there isn’t an actual 8-year electronics death clock out there running somewhere in ULs labs.

The 11 year old somewhat-computerized stereo bluetooth nav system in our SUV still works.. mostly. We all have 30+ year old consoles that still work. I have a 2009-era iMac that works. But, I suspect that these are more attributable to Honda, Apple and Nintendo using quality parts or over-speccing components.

When these tiny components fail, it usually isn’t a complete failure either. The preset station ‘4’ on my 08 Mercedes Convertible no longer recognized button presses because the surface-mount-resistors around the button had failed. That Harmon Kardon stereo would still Jam, though. In fact, I think I miss that radio more than the car… (focus, Bill.. focus..)

When the SUV’s bluetooth controller failed, I was able to fix it once with a heat gun and eventually replace it for about $400.

 
They-dont-make-em.jpg

Increasingly, I’m feeling the trueness of that “they don’t make them like they used to”, sentiment being muttered at barber shops and country breakfast buffets across our fair land.

And it’s true: They don’t. They use machines to make other machines that are (in some ways) millions of times more complicated than what was necessarily 2 decades ago.

That’s progress. I’m not railing against it but I want to have fair expectations.

 

Bringing it Back to The Hobby

Every new Stern Pinball designed after 2015 has Single Board Computer (SBC) in the back box. It handles lamp control, registers switch hits and powers coils through a modular system of ethernet-connected peripheral PCBs (node boards).

Stern SPIKE:
Proprietary SBC in the Back, Many Proprietary PCBs in the Cabinet

 
Stern's SPIKE 2 System (in the backbox)

Stern's SPIKE 2 System (in the backbox)

Typical Stern Node Board.

Typical Stern Node Board.

Typical Node Board Placement

Typical Node Board Placement

CGC/PPS:
Proprietary SBC in the Back, Big n’ Honkin PCB in the CAB

Every new CGC/PPS Pinball includes a similar design and adds a half-playfield-length board beneath the playfield that handle coil control, switches, controlled lamps & flashers.

Typical CGC Backbox (SBC Controller)

Typical CGC Backbox (SBC Controller)

Typical CGC under-playfield board

Typical CGC under-playfield board

Tiny little mass-produced surface mount components in all of it.

 

JJP:
Blended / Modular Approach

Every new JJP pinball that was ever made includes an actual PC connected to an IO controller to make all of that pinball magic happen.

HobbitPC.jpg

Hobbit and Woz both used a commercially-available MSI ATX motherboard, ATX power supply mixed with a proprietary IO Controller and sound card. Notably, the IO Controller still has fuse-able circuits and transistors. I credit JJP for these decisions. Those modular designs with open components will-be repairable in 20+ years.

What about The Others?

I personally wouldn’t be too interested in owning a pinHeck-based Spooky title. Nothing against the electronics design so much as the pinball designs themselves didn’t jive with my tastes. I actually like the modular nature of Danesi’s Total Nuclear Annihilation but the clear-coat curing issues around posts scare me off from that game. I don’t know enough about Rick & Morty’s electronics to comment as of this time.

Closing thoughts

Pinball machines are computers now. You know what that means, right? That means they inherit all of the bullshit you get from computers. Boot times, patches & hotfixes and boards that will be increasingly difficult for a hobbyist to repair. Somewhere, someone is having a SCRUM standup related to a pinball software release. shudders poor bastards

Nearly every arcade machine from the 2000’s on have been some form or another of computer. Sometimes they look more like repackaged console systems and sometimes they look more like traditional PCs. But make no mistake - they are PCs. You know what that means right? You shouldn’t be surprised by the failure of the Big Buck Hunter hard drive or Hydro-Thunder PC.

 
 
Mario Kart GP/ GP2, Maximum Tune 3, Tekken 5 & order mid-2000’s arcades are running Tri-Force hardware that is very close to GameCube.

Mario Kart GP/ GP2, Maximum Tune 3, Tekken 5 & order mid-2000’s arcades are running Tri-Force hardware that is very close to GameCube.

Offroad / Hydro / Arctic Thunder looks something like this

Offroad / Hydro / Arctic Thunder looks something like this

On the upside.. It also means you inherit all of the cool stuff attributable from having a computer under the hood. Expandability through software, richer audio-visual experiences, fine-grained control of components (like pinball coil strength or video game joystick sensitivity), social gaming features and a more-connected future.

It also means that we might be entering into a world where our $1000-$15000 collectible toys have a comparably shorter lifespan.

Those tech-stack dog years might not be kind to us.

The next time I buy a “modern” or NIB pinball, I’m leaning towards a CGC/PPS Remake or JJP for the overall build quality and design. Either way, before I buy another NIB or modern pinball (or arcade for that matter) I’m going to ask myself some questions.

  • How proprietary are the components of the game?

  • Are parts available today and at what cost?

  • Is it likely that this company will be operating in 10 years?

  • If the company folds, how likely is it parts (or parts analogs) will be available?

  • Do they have a customer-support reputation, post-sale?

  • What is my average game-retention time? Is the game likely to “age out” in that window?

Yoda.jpg

It is fair to note that in the case of pinball boards, boutique electronics companies filled the void long after the original manufacturers left the space. Rottendog, Alltec, Great Lakes Modular & others stepped up to build replacement boards that blended the old design with modern components. (With subjectively mixed results compared to the originals.) This was made possible through availability of original schematics, ample demand in the community and an increasingly empowered Maker’s culture.

Who knows, maybe 20 years from now we can buy electronically guided, micro-scale hot air rework stations for the price of a shovel at Home Depot, making these boards reasonably repairable again.

Sometimes, the best way to deter obsolescence is to go modular like JJP did. All-in-one solutions will only last as long as the weakest component. For those folks that bought a high end SONOS speaker system, the traditional approach of a receiver + speakers + components would probably have aged better.