Category Archives: technology

Whales, Tails and How Far to Trust AI

How can you train an AI?

By AI, we're referring to "artificial intelligence" systems, which are a special class of machine learning computer programs that are increasing showing up in some pretty amazing applications. Whether its generating an image based on text you enter or nearly instantaneously writing the equivalent of a school report on a particular subject, AI systems are leaving the world of science fiction and becoming today's reality.

But how do their developers train these systems to do these things?

Last year, Matt Parker visited Antartica, where he learned how to apply maths to identify specific humpback whales. The following 22-minute video describes how the mathematical methods developed for advanced image recognition made it possible for him to use an Excel spreadsheet to identify a specific whale he photographed swimming off the north coast of Antartica*.

Clearly, AI can deliver impressive results, but how far can you trust those results?

One area where photo-recognition AI systems could make a real impact is in radiology, where such systems could potentially diagnosis serious health conditions much more quickly at much lower cost than can be done by professional radiologists.

A recent study published in the British Medical Journal (BMJ) asked if AI could pass the Royal College of Radiologists' board examination. Spoiler alert: It couldn't, where why it couldn't tells us something about the limitations of these AI deep maching learning systems. Chuck Dinerstein of the American Council on Science and Health summarizes the study's main findings, in which the performance of AI-trained systems and human radiologists were compared (emphasis ours):

First, the obvious, with two exceptions, humans did better than the AI on diagnosis where both had been trained; when unfamiliar pathology was introduced, AI failed across the board. Second, while the humans fared better, theirs was not a stellar performance. On average, newly minted radiologists passed 4 of the ten examinations.

“The artificial intelligence candidate... outperformed three radiologists who passed only one mock examination (the artificial intelligence candidate passed two). Nevertheless, the artificial intelligence candidate would still need further training to achieve the same level of performance and skill of an average recently FRCR qualified radiologist, particularly in the identification of subtle musculoskeletal abnormalities.”

The abilities of an AI radiology program remain brittle, unable to extend outside their training set, and as evidenced by this testing, not ready for independent work. All of this speaks to a point Dr. Hinton made in a less hyperbolic moment.

“[AI in the future is] going to know a lot about what you’re probably going to want to do and how to do it, and it’s going to be very helpful. But it’s not going to replace you.”

Here's the kicker according to Dinerstein:

We would serve our purposes better by seeing AI diagnostics as a part of our workflow, a second set of eyes on the problem, or in this case, an image. Interestingly, in this study, the researchers asked the radiologists how they thought the AI program would do; they overestimated AI, expecting it to do better than humans in 3 examinations. That suggests a bit of bias, unconscious or not, to trust the AI over themselves. Hopefully, experience and identifying the weakness of AI radiology will hone that expectation.

Like any human expert, AI has limitations. Identifying and knowing what those limitations are will be key to determining how trustworthy they are. In the case of health care, as the example from radiology makes clear, it could be your health that's on the line if you blindly put more trust into a system than it deserves.

Reference

Shelmerdine, S.C.; Martin, H.; Shirodkar, K.; Shamshuddin, S.; Weir-McCall, J.R. "Can artificial intelligence pass the Fellowship of the Royal College of Radiologists examination? Multi-reader diagnostic accuracy study." BMJ 2022; 379. DOI: 10.1136/bmj-2022-072826. (Published 21 December 2022).

* If you know your geography, you already knew every coast of Antartica is the north coast!...

Inventions in Everything: The Snowball Shotgun

With big winter storms having dumped tons of snow across much of the United States, what to do with all that snow has become a challenge for many Americans. What they won't be doing however is using J.W. O'Dell's 1952 invention, for which he was awarded with U.S. Patent 2,607,333: The Snowball Gun.

It's not because the idea of a snowball gun isn't popular. Amazon alone carries a least a dozen different kinds of snow-propelling toys, including blasters, slingshots, cannons, throwers, et cetera. However they all share one key characteristic. They are unmistakably toys.

By contrast, Figure 1 of O'Dell's snowball gun patent clearly envisions what we'll describe as a short, double-barreled snowball shotgun.

U.S. Patent 2,607,333 Figure 1

That's the kind of toy that could convince store clerks to hand over the contents of their cash registers. Not that O'Dell envisioned such an outcome. The patent's abstract indicates he saw his concept of a snowball gun as a source of good clean fun.

This invention is for a toy gun for projecting snow pellets. The object of the device is a toy designed to be charged with loose snow and to form and project the same in the form of pellets. A further object of this device is to provide the user with considerable action by providing two parallel barrels to provide harmless amusement.

Then again, his invention was conceived in a very different time. Check out the following video with some select toy commercials from the 1950s and 1960s.

With that kind of big-toymaker marketing, the snowball shotgun would hardly have stood out in the market.

Inventions in Everything: The Archives

Ready to sample more of the most creative designs and patents the Inventions in Everything team has explored? Our archives celebrate inventions ranging from the whimsical to the inspired in reverse chronological order!

Mars’ GDP Triples in Martian 3Q

Six Earth months, or rather, one Martian quarter has passed, which means it is time once again to estimate the red planet's gross domestic product!

Mars... I can't believe I'm back on Mars.

As the Martian economy completes the third quarter for which we've estimated its planetary GDP, we find it is humming along after having run into a rough patch during the previous Martian quarter. Starting from Earthdate 7 July 2022, robotic mining operations have proceeded without significant technical interruption and the Perseverence rover has resumed collecting rock core samples. This activity followed nearly four Earth months with no sample collection when the rover traveled from the Octavia E. Butler Landing/Ch’ał outcrop where it drilled its previous core sample to the sample-rich geology of the Delta Front within the Jezero Crater.

The rover drilled nine samples during the Martian third quarter, with the samples placed into inventory pending their planned export to Earth. With a total of seventeen samples, we now have the following revised GDP estimates for Mars' first and second Martian quarters and an initial estimate for the third Martian quarter:

Martian GDP Estimates (Constant 2021 U.S. Dollars)
Martian Quarter First Quarter Second Quarter Third Quarter
Approximate Earthdates 12 Jul 2021 - 31 Dec 2021 1 Jan 2022 - 21 Jun 2022 22 Jun 2022 - 11 Dec 2022
Estimated GDP $494,430
($110,780 - $878,080)
$296,658
($66,468 - $526,848)
$889,974
($199,404 - $1,580,544)
Revision Level Third Second Initial

We estimate Mars' GDP tripled its previous quarter's revised GDP total during the planet's third quarter, falling just shy of $900,000 in terms of constant 2021 U.S. dollars. At present, we're valuing the two regolith samples collected by the Perseverance rover the same as the seven rock core samples it collected during the Martian third quarter in the absence of better valuation data for it. We're also assigning a minimal value to the Martian atmospheric sample the rover collected and stored during the Martian first quarter (in August 2021), given the very low density of the Martian atmosphere and the low premium of $50/gram given to Martian meteorites on Earth determined to have trapped atmospheric content within them.

Looking ahead to Mars' future export economy, NASA and the European Space Agency reached an agreement to establish a sample transfer depot at the Three Forks location within Mars' Jezero Crater. The Perseverence Rover will cache its full inventory of rock and atmospheric samples at the Three Forks Depot before their future export.

Unfortunately, the government space agencies have pushed back their planned export operations, with deliveries to Earth now delayed from 2030 to 2033. They've also changed their plans for how the samples will be recovered before their transportation to Earth:

... the Mars Sample Return campaign will no longer include the Sample Fetch Rover or its associated second lander. The Sample Retrieval Lander will include two sample recovery helicopters, based on the design of the Ingenuity helicopter, which has performed 29 flights at Mars and survived over a year beyond its original planned lifetime. The helicopters will provide a secondary capability to retrieve samples cached on the surface of Mars.

The ESA Earth Return Orbiter and its NASA-provided Capture, Containment, and Return System remain vital elements of the program architecture.

With planned launch dates for the Earth Return Orbiter and Sample Retrieval Lander in fall 2027 and summer 2028, respectively, the samples are expected to arrive on Earth in 2033.

The following three-minute video describes the final stage for how the samples will be delivered to Earth during the final stage of their transport.

Previously on Political Calculations

Inventions in Everything: Santa’s Ski Resort Christmas Ornament

It's that time of year once again! Families are already focusing on the details of decorating to celebrate Christmas, and how to decorate their Christmas tree in particular.

As you hang your family's ornaments this year, you've probably noticed most of them aren't very exciting. And to be sure, they're not. If you think about it, their only function is to hang there on your tree. Wouldn't it be nice to have Christmas ornaments that feature a lot more action to liven the holiday spirit up.

Inventors Marc Segan and Michael Newsome were thinking along those lines back in the early 1990s. Their idea was to transform a large section of a Christmas tree into Santa's ski resort, complete with a ski run and lift, where a miniature figure could ride the lift up to the top, then slalom through the branches and around the tree back to the bottom before repeating the cycle. For their innovative thinking, they were awarded U.S. Patent 5,279,871 on 18 January 1994. Here's Figure 1 from the patent, which we've colorized to make the various components stand out:

U.S. Patent 5,279,871 Figure 1

Segan and Newsome describe how they brought action to holiday ornaments with their invention:

The present invention is a Christmas display resembling a ski-lift and ski-slope, for use in conjunction with a Christmas tree. The display comprises a track having a first end and a second end, wherein the first end is at a higher elevation than the second end, and a lift disposed between the first end and the second end. The device also comprises a plurality of figurines having a base configured for slidable movement along track from the first end of track to the second end of track whereby the lift transports the figurine back to the first end of the track in a continuous manner.

The figurine (30) in the patent illustration is a miniature Santa on skis! Just imagine him skiing around your tree through the whole holiday season, celebrating the holiday at the ski resort on your Christmas tree. The only way you could have more action in a Christmas setting with with a scale model of the Nakatomi Plaza building and a miniature reenactment of Die Hard.

From the Inventions in Everything Archives

Here's a selection of Christmas and tree-related inventions the IIE team has previously featured:

Inventions in Everything: Santa’s Ski Resort Christmas Ornament

It's that time of year once again! Families are already focusing on the details of decorating to celebrate Christmas, and how to decorate their Christmas tree in particular.

As you hang your family's ornaments this year, you've probably noticed most of them aren't very exciting. And to be sure, they're not. If you think about it, their only function is to hang there on your tree. Wouldn't it be nice to have Christmas ornaments that feature a lot more action to liven the holiday spirit up.

Inventors Marc Segan and Michael Newsome were thinking along those lines back in the early 1990s. Their idea was to transform a large section of a Christmas tree into Santa's ski resort, complete with a ski run and lift, where a miniature figure could ride the lift up to the top, then slalom through the branches and around the tree back to the bottom before repeating the cycle. For their innovative thinking, they were awarded U.S. Patent 5,279,871 on 18 January 1994. Here's Figure 1 from the patent, which we've colorized to make the various components stand out:

U.S. Patent 5,279,871 Figure 1

Segan and Newsome describe how they brought action to holiday ornaments with their invention:

The present invention is a Christmas display resembling a ski-lift and ski-slope, for use in conjunction with a Christmas tree. The display comprises a track having a first end and a second end, wherein the first end is at a higher elevation than the second end, and a lift disposed between the first end and the second end. The device also comprises a plurality of figurines having a base configured for slidable movement along track from the first end of track to the second end of track whereby the lift transports the figurine back to the first end of the track in a continuous manner.

The figurine (30) in the patent illustration is a miniature Santa on skis! Just imagine him skiing around your tree through the whole holiday season, celebrating the holiday at the ski resort on your Christmas tree. The only way you could have more action in a Christmas setting with with a scale model of the Nakatomi Plaza building and a miniature reenactment of Die Hard.

From the Inventions in Everything Archives

Here's a selection of Christmas and tree-related inventions the IIE team has previously featured: