The Future of Pen Input Tech

Technology General Discussion
, ,
1

After reading through the many wonderful stories in the “What got you into Tablet PCs?” thread, I was wracking my brain trying to think of my own. But it dawned on me that there was no initial epiphany for me—I just always thought of ‘computing’ as tablet use.

Even today, I can’t quite shake the feeling that keyboards bring you subtly apart from your work, while the pen, closer to it. Still perplexing is the modern concept of the “notebook” computer: is it not strange this means a laptop, instead of something you hold in the hand and write on?

The trajectory for this most expressive and ubiquitous medium has always seemed oddly stunted in tech. On the ex-forum, I remember sharing this video with excitement:

Finally! I thought, some long-overdue recognition of the tactile dimension of pen input, a recognition of the hand-eye connection in learning and ideation.

That was 10 years ago. Now the closest thing we have is the Surface Slim Pen haptic motor—so obscure that I wonder if even the devs in the initial demo remember putting it in.

What gets me is not the lack of hardware advancement per se, but the slow withering of any seeming impulse in the industry to innovate:

Does anyone remember the Sensu Brush?

image
image725×480 69.8 KB

A simple passive stylus, that made waves for its unique goal of bringing a soft bristle feel to the tablet experience. That came out before the era of the Apple Pencil.

Now as active-capacitive technology has matured, we don’t hear of any company trying to add extra dimensionality to the iPad digitizer. Where is the industry’s drive to go beyond and explore, all the passion that fuelled the initial iPad art rage?

We’ve long joked about the 30-year stagnation in Wacom EMR tech, but it’s evident that even the new entries from Huion and XP-Pen only seek to imitate. For better or worse, we are stuck in a pen input time capsule.

So I put it to you guys, is there room for meaningful advancement in pen tech?

If not, why do you think they digital pen never caught on, even when most people still used pad and paper in the 80-90s? And paradoxically, if the pen really is a relic, why has it proven so difficult to kill off even in 2023?

2 Likes
2

Thanks for sharing this.

I still believe it is two things:

  1. because the software capability has never met the right hardware. I’ve written before about Franklin Covey’s Tablet Planner that came out with the original Tablet PCs. You could keep your paper planner experience and link it to your outlook calendar, recognize your handwriting as text or not, and even the to do lists were “dynamic” - cross it off and it moved to the bottom of the list. The hardware was too slow and battery life too short, so it never really caught fire.

  2. Second, the PC/desktop/keyboard interface still drives most business productivity settings and inking spreadsheets is just not as efficient.

Not sure why it never caught fire with the art crowd. Resolution?

2 Likes
3

I think the main thing with the art crowd has been more to do with hardware and software working together as well. For the longest time, if you were a serious artist, and wanted to do it digitally, you used a large Wacom tablet or pen display connected to a fully powered PC or Mac. Photoshop wasn’t available on anything other than PC or MacOS, and if you were a professional, Photoshop is what you used, or at least one of the Adobe art products. Mac didn’t offer a pen for anything other than iPad, and Windows pen offerings couldn’t match Wacom for accuracy needed in the art world. There were a few exceptions over the years that offered EMR pens, but mainly they were before 2 in 1s became truly portable and gave truly all day battery, and still didn’t compete with a desktop’s power, and couldn’t provide the screen real estate that a pen display could.

Even now, with Photoshop and Illustrator on the iPad, it’s an uphill battle for a true professional artist to use that as a primary device. Android is even worse in that regard, and Samsung, the lone holdout for Wacom for Windows tablets still doesn’t really compete with a pen display and desktop setup.

I would love advancement in pen tech for art though. A realistic brush with multiple pressure sensitive bristles that each provide their own info for pressure and location to the tablet would be amazing. It also sounds extremely hard to program and produce and even if it sold like wildfire, would likely not be very profitable.

3 Likes
4

There used to be something that would have been perfect to be paired with the sensu brush. Remember the long dead Nvidia Direct Stylus tech? It was the technology baked into the also long dead Nvidia Shield tablet series, where you could use a simple capacity stylus to “fake” pressure sensitive.

It had palm recognition and also the ability to calculate the contact area of the stylus on the screen, the stylus had a soft pointy tip so if you press harder, the tip is deformed and make bigger contact area and make thicker lines. Someone tried sensu brush on it, and the stylus made varied like exactly based on contact, unlike a normal touchscreen that only made an uniform line. In their custom Dabbler app, they could also do pressure opacity emulation based on contact area. And as said in this video, they did it for half the cost of a Samsung tablet.

This could have been a perfectly affordable way to add decent stylus to budget phones, but unfortunately the tech never take off. Despite the Terga chip being used in a few devices, they never enable this tech on anything new and the tech had been long dead. The lack of stylus support in the Nintendo switch was especially a missed opportunity considering they used terga chips, and people had to resort to clunky dongle third party stylus for art game.

5 Likes
5

Man, the 90’s was a fun time…all the possibilities of what could have been for pen-based computing.

1 Like
6

It has always been about the right software - much of which never materialized or got optimized.

7

Decided to shift gears and pontificate on this topic, if you’ll indulge me a little. :wink:

I’m still baffled at how pen input got completely sidelined in the 90s. Jake Weidmann’s TEDtalk reminded me how pen input has always been the facilitator of human creativity:

If computers are the fundamental tool of creation in the 21st century, surely the pen would have a place.

Yet countless attempts across multiple operating systems and form-factors still has produced an awkward UX in modern machines: at odds with the keyboard & mouse, yet not fully integrated with the mobile touchscreen.

Again baffling…as pen and tablet is arguably humanity’s first physical “UI”.

1 Like
8

@SeanN
9 years ago
Very unfortunate how this turned out. Palm rejection is very hit or miss, and the eraser mode activation is not predictable. A k1 variant with more cycles available to dedicate to the software detection algorithm may produce very active stylus like results, but currently, the tegra note is not really a competitor for even the aging note 8. I dont regret my purchase, but for art and writing, workarounds aplenty are still needed.

It seems it wasn’t as great as it seemed. That Nvidia dropped it and Nintendo didn’t take it up says quite a bit.

1 Like
9

Care to speculate why tablet OSes didn’t evolve, before there was mobile?

Even before there was even a screen on a phone, tablets had hype, innovative UI and even the stylus going for it, coming from PDAs.

10

I do think mobile came about partly because of that. Some people wanted a tablet with pen input. Almost everyone could use a mobile phone. Maybe it was foresight, maybe it was just dumb luck. In order for tablets to become nearly as popular as mobile phones, they needed to be as useful as mobile phones. They just aren’t, and never were going to be. The only way it was going to work would have been for the hardware to be much more advanced than it was at the time. A true desktop/laptop replacement that was innovative and easy to use needed better performance, better, battery, and a very intuitive user interface. Even if the hardware could have managed that, I think Windows 8.1 is proof enough that people weren’t ready for the interface.

1 Like
11

I had the occasion to visit my local hospital’s emergency room this weekend (nothing ever happens during the week in my family, only the weekend, when no one is there - alls well that ends well). I am amazed at how much of the nurses’s work is done by the IPhone. Only a couple of years ago, that work was being done by dedicated tablets or bar-reader devices. In large part, tablets have been left behind in the field. The only people working from full workscreens using dedicated terminals are the people at intake and at the nurse’s station. Virtually everything else uses the phone as input device.

2 Likes
12

My wife homeschools our 4 kids, and aside from the actual workbooks she uses for curriculum, most of her material she uses her cell phone for. She prints out sheets for them to do from the phone, sets up assignments, etc. It’s only when she really needs to do some major editing in Word that she pulls out her laptop. The kids still do a lot of their work on a laptop, but it’s managed from a phone. My wife’s laptop’s primary use is a portable TV more than anything else.

For my own use case, if I didn’t do digital art, I’d probably be happy with just a mobile phone for everything non-work related. I’d probably want a larger one than I have now, but otherwise, I think I’d be pretty happy with just that setup. This is also why I think a fully fleshed out Dex setup could be the way of the future. Docking a phone for those only as needed use cases and when you really want a keyboard to type out anything longer would be pretty great for most users. Even enterprise solutions would just need a phone if their apps had proper Dex support.

13

Interesting point. Tablet came ahead of their time, whereas smartphones came along just at right time.

One possible counterpoint, is that tablets still have a core usecase (despite mobile and desktop taking up the ends). This is why iPads are still around and you see plethora of Android tablets coming out of China. Specialized OSes evolve for devices like cameras, cars, smarthome devices; I don’t see anything preventing tablets from also evolving a particular UX paradigm.

14

I don’t disagree that there is a market for tablets. I do think one of the reasons they’re affordable is the use of blown up mobile OSes though. If the price went up too high, IDK that we would have as large of a market for them. Cameras, and other devices, etc, don’t have the problem of the app store, and honestly, a lot of them are still modified versions of Android, so still mobile OSes at their heart. There’s a lot of savings by using what is already available and already proven to work for those types of scenarios from having to develop new OSes each time. And I can’t think of any developers who would find the market share worthwhile to build custom apps for such niche devices for yet a smaller portion of their users to then download said custom apps. The cost of development time would surely make the app cost astronomical in comparison.

I think on the flip side, a sub $200 Android tablet is just cheap enough that it covers a large market for people who honestly either would otherwise just use their phones, or would otherwise just use a laptop/chromebook.

I think about people like my parents, who have a laptop, and their phones, but aren’t all that tech savvy and see a larger version of their phone as an easy to use device. They are more than willing to buy a cheap tablet to get a larger screen. If they couldn’t find something cheap enough, they would just use their phones and be perfectly happy with it. Honestly, they’ve probably gone through 2-3 tablets in the last 5 years, and they all just sit around the house, while their phones are still their primary devices.

I guess what I’m saying is, I think in the current state, phones are the only reason tablets even have a market. I think their cheapness makes them more purchased than they are used by many. They almost inevitably sit and gain dust in most houses. If the expense was put in to make them useful, I don’t think the current market would support that cost.

15

I think you might be underestimating the premium tablet segment, which only seems to be attracting increasing interest by the sheer quantity of OEMs (eg. Lenovo Tab P Pro, Xaiomi Pad Pro, Honor Pad X, Oppo Pad Air).

So I say tablets can stand on their own, by the nature of their form-factor alone. We just need the right software.

Just as I was writing this, I just thought of something—maybe e-ink could be the saviour here:

E-ink devices have a focused enough usecase to drive independent UX development from smartphones. They are already well-situated in the physical notebook paradigm, and the userbase almost defacto expects pen support, enough that that almost all e-ink note tablets come bundled with an EMR stylus.

Perhaps most salient, is they are focused on information and productivity, something the generic Android tablet never could convey.

16

Don’t get me wrong, I think there is a really good use case for tablets outside of just pen input uses. But I think a lot of the perceived value is from how heavily people use smart phones in general. Back in the day, when smart phones were first becoming a thing, I think people looked at their laptops/desktops, and thought, “wow, I can do laptop things on my phone!”. I remember being in that camp anyway, thinking how cool it was that I could edit actual documents, etc. Internet browsing, videos, games, and entertainment in general were painfully slow, but productivity on the go! That was awesome. Now that people can use their phones for all that entertainment stuff, plus everything else, they’re looking at it from the opposite side. They aren’t looking for portable laptop shrunk down, but phone blown up. That’s why I say phones are the reason tablets have a market.

I think that’s part of why Samsung is able to sell the S9 + and Ultra now. More and more people are used to the Android interface for everything and are happy to translate that experience into productivity on a larger scale. I tried to find some market trends to figure out how much of the market is run by Samsung’s higher end devices, and Apple’s higher end devices, but most actual numbers seem to be behind paywalls.

My point though is, I think because of IOS and Android on phones, more and more people are using premium tablets such as the S9 and iPad air for bigger screen things. I don’t think a new OS will help tablets to stand on their own. I think the familiar mobile OS is what makes them attractive in the first place.

Now, if they gain more popularity, as trends seem to be predicting, with the rise of work from home, etc, maybe we’ll see a bridge 3rd OS. Greater development of things like Dex, stage manager, or something similar that meets in the middle. I don’t know for sure. I personally just don’t see it going that way. But that’s just my opinion. I’ve been wrong once or twice in my life before. Definitely could be wrong again now.

17

What do you think of the e-ink notetaker angle? The new e-ink devices do stand on their own in terms of UX, distinct from the typical tablet.

It wouldn’t be so much a bridge OS, rather a separate development paradigm of the “digital notebook”, in many ways returning to the roots of tablet computing.

18

I remember when the first eInk device came out 20 something years ago. I was super excited about it then and thought it would take over that segment as digital notebook then. I think they’re very cool, and have a ton of potential, but it’s been a very slow crawl to get where they are now, and so many tablets do everything they do without the compromises. IDK, maybe I’m just cynical on this one, but I don’t know that eInk devices will ever live up to their potential from a market share standpoint. Sure, lots of people use them for e-readers now, but maybe partially because so many of them have had their own standalone OSes, very few people use them to take notes of any sort. The lack of flexibility kind of leads to them not being all that useful in that way. For the longest time you could only access your notes on that device. I think that’s why so many are now just moving towards having the play store available on them. Being unique is cool and all, but being familiar and working across devices is inevitably what will work.

1 Like
19

I think the challenge is friction between that independent Os development and the information stored in desktops and phones. A third set of UI to learn and negotiate data access through is a big barrier. If Boox really gets traction, maybe it’s the model.

1 Like
20

Just curious, but what would you consider to be full “potential” or real “traction”?

E-ink has been growing quite steadily and anecdotally, many users prefer that it isn’t as congested as a tablet/mobile OS. Pursuing explosive growth can often derail development and in this case, turn off the user base.

I would say slow and steady development, with a unique pen & tablet UX is the way. :wink: