10 Incredibly Creative Things You Can Do With OpenAI's New GPT-4o

OpenAI just announced GPT-4o with promises of better human-to-AI interaction. Let’s dive into what makes it the best model release of 2024 so far.

John Angelo Yap

Updated May 20, 2024

A human talking to an OpenAI robot visualized, generated with Midjourney

A human talking to an OpenAI robot visualized, generated with Midjourney

Reading Time: 6 minutes

The moment that AI was no longer the talk of the town was the moment that we truly entered the AI era. It’s become so naturalized to our society to the point that it’s integrated into our education, work, and everyday life. 

However, one thing that’s limiting our access to AI is the lack of human-computer interaction support. Only a handful LLMs offer multimodal support, and even fewer do it free or accurately. OpenAI might’ve just solved that issue.

In this article, I’ll be discussing briefly what it is and some of my favorite use cases so far of this model. 

Disclaimer: All video links provided below are courtesy of OpenAI.

What is GPT-4o?

GPT-4o (“o” stands for omni) is OpenAI’s newest LLM. It’s made to create more natural human-computer interactions by expanding its multimodal capacity and supercharging its nuance. It has an average response time of 320 milliseconds, which is close to the human response time.

Here are a few nifty ways to use it:

Real Time Translation

Ever find yourself lost in a foreign country without any means to communicate? OpenAI has you covered.

One of GPT-4o’s most significant features is its multilingual support. Along with multimodal inputs, ChatGPT can easily translate from one language to another faster and almost as accurately as any human translator. With a turnaround time of about 232 milliseconds for audio, ChatGPT with 4o can be your best friend whenever you’re traveling or speaking to someone not fluent in your language.

Meeting AI Assistant

Meetings can be draining. You never know when you’re dozing off or when your attention’s going elsewhere. 

With GPT-4o, you can always be on top of things by using it as an AI assistant for meetings. It can act as a guide whenever someone asks you a question, take minutes of the meeting to revisit later, or clear up things when it gets confusing.

Harmonize

This is one of the craziest things I’ve seen from an AI. We’ve all become accustomed to AI taking inputs in different forms, but I’ve never seen a truly multimodal AI to the point that it can create beats, adjust tone, and actually harmonize to create music. What makes it better is that you can give it additional context as they go along to nail the sound you’re looking for.

Complete Math Assignments

Okay, I know an AI that can do assignments isn’t out of the norm today — but wait until you see what GPT-4o can do.

This new model can answer mathematics questions in real-time. Using its new desktop app, GPT-4o can take questions in the form of text, images, or video, and act like a tutor by giving you the information you’re looking for while explaining how it came to that answer. You can even narrow down your questions in real-time by feeding it further context.

From what I’ve seen, GPT-4o also has greatly improved mathematical reasoning compared to previous GPT models. 

Perfect Text Generation

Gone are the days of imperfect text generation from DALL-E 3. With GPT-4o, ChatGPT can now write paragraphs of text in an image with little to no mistakes. You can even tweak how you want the text to look. Let me tell you, as someone who’s using AI image generators daily, this is insane.

Character Reference

Midjourney may be the first AI image generator to release their character reference feature, but I think OpenAI has them beat on accuracy. Using GPT-4o, DALL-E 3 can now create consistent characters based on an input or a previous generation.

Summarize Information From A Video

Similar to the meeting AI assistant, GPT-4o can also transcribe and summarize an entire video as input. To do this, you must input a video directly and not a link. For example, here’s a snippet from OpenAI’s showcase of the model’s input and output:

Original ChatGPT Text

The ketogenic, or "keto," diet is a low-carb, high-fat eating plan that puts the body into a metabolic state called ketosis. In ketosis, the body burns fat for fuel instead of carbohydrates. By drastically reducing carb intake and replacing it with healthy fats, the keto diet can promote weight loss and improved blood sugar control. However, it's a restrictive diet that requires diligent tracking of macronutrients and could increase risk of nutrient deficiencies if not properly planned. While effective for some, the keto diet may not be suitable long-term due to its elimination of many nutrient-dense foods. As with any diet, it's crucial to consult a healthcare professional, especially for those with underlying medical conditions. With commitment and guidance, the keto approach can be a useful tool for certain health goals.

Blog Title: “The Surprising Risks of Keto Diets: Gut Health, Heart Disease, and More”

Nutrient Deficiencies on Keto Diets

Ketogenic diets, which severely restrict carbohydrate intake, have been used for decades to treat certain cases of pediatric epilepsy. However, a closer look at the safety data reveals some concerning issues. One of the obvious problems is the risk of nutrient deficiencies. Inadequate intake of 17 essential vitamins and minerals has been documented in those following ketogenic diets. This is a particularly important concern when dieting, as you may be taking in less food overall.


In fact, ketogenic diets are so nutritionally sparse that one assessment estimated you'd have to consume more than 37,000 calories per day to get a sufficient daily intake of all essential nutrients. This is in stark contrast to more plant-based approaches, which tend to be much more nutrient-dense. As the editor-in-chief of the Journal of the American Dietetic Association put it, "What could be more nutrient-dense than a vegetable diet?" Choosing a healthy diet may indeed be easier than "sticking 50 sticks of butter in your coffee." The vitamin and mineral deficiencies associated with keto diets can be mitigated with supplements, but there's another concerning issue: the lack of prebiotics. Prebiotics are the dozens of types of fiber and resistant starches found concentrated in whole grains and beans. This lack of prebiotics can have a whole array of negative consequences for gut health.


Keto Diets and Gut Health

Not surprisingly, constipation is very common on keto diets. But the problems go deeper than that. Starving our gut microbiome of prebiotics can lead to detrimental changes in gut flora. Ketogenic diets have been shown to reduce the richness and diversity of our gut microbiome, which can be detected within just 24 hours of switching to a high-fat, low-fiber diet.

The lack of fiber starves our beneficial gut bacteria, while the saturated fat from the diet can make its way down to the colon, potentially resulting in weight gain, increased gut permeability (leaky gut), and pro-inflammatory changes. For example, keto diets have been linked to a drop in beneficial Bifidobacterium and a decrease in overall short-chain fatty acid production, both of which would be expected to increase the risk of gastrointestinal disorders.

Keto Diets and Heart Disease

When it comes to the heart, the news isn't much better. Looking at low-carbohydrate diets and all-cause mortality, those who eat lower-carb diets suffer a significantly higher risk of dying prematurely. And it matters whether the low-carb diet is animal-based or plant-based. Eating more of an animal-based low-carb diet was associated with a 50% higher risk of dying from heart attack or stroke, but no such association was found for lower-carb diets based on plant sources.

Cholesterol production is directly correlated to body weight, and every pound of weight loss is associated with about a one-point drop in cholesterol levels. However, the beneficial effect of keto diets on LDL ("bad") cholesterol is blunted or even completely neutralized, counterbalancing any potential gains. And within just three hours of eating a high-saturated-fat meal, you can see a significant impairment of artery function, even with substantial weight loss.

Keto Diets and Bone Health

The negative effects of keto diets don't stop there. They may also impact bone health. Tune in next time to find out how keto diets could be bad for our bones.

In the meantime, for those looking to improve their heart health, a diet centered around whole, plant-based foods may be the way to go. As explored in this video, lifestyle approaches can be safer, cheaper, and more effective than drugs at treating the underlying cause of heart disease.

For more on the latest in nutrition research, be sure to subscribe to the NutritionFacts.org newsletter. And if you have any questions about this video or the topic, feel free to leave them in the comments section below.

As someone who watched the video in its entirety, I can confirm that GPT-4o didn’t miss any key information. This is a huge evolution compared to its previous iteration.

Transcribe Illegible Text

Have you ever unearthed an old piece of paper with text you can barely — if at all — read? Let OpenAI do its magic.

GPT-4o combines multimodal support with enhanced natural language processing to turn illegible handwriting into string using contextual understanding. Here’s an example from Generative History on Twitter:

Create A Facebook Messenger Clone

I was browsing Twitter last night and found what might be the biggest case for GPT-4o’s improved capabilities. Sawyer Hood from Twitter wanted to test this new model by asking it to create a Facebook Messenger clone. 

The result? It worked. Not only that, but GPT-4o did all of these in under six seconds. Sure, it’s just a single HTML file — but imagine the implications of this in front-end development in general.

Understand Intonation

And now, we’re down to what I consider GPT-4o’s biggest accomplishment, though some might not agree. In the past, LLMs have always taken what we feed into them at face value. They rarely consider our tone or phrasing in processing our inputs. 

That’s why I’ve always considered models that can do sarcasm as science fiction. Well, OpenAI just proved me wrong.

All Said And Done

There's a lot of talk about Gemini, Claude, and other LLMs potentially passing OpenAI in terms of nuance and features. Well, this is OpenAI's answer to them.

GPT-4o is the first model I've seen that feels truly multimodal. Not only that, but it's also solved some of the issues that plagued GPT-4 in the past in terms of being lazy and lacking in nuance. 

OpenAI is a company that’s been way too familiar with controversies in the past, but I have a gut feeling that people are going to forget those soon with GPT-4o. I can’t wait to see where OpenAI takes LLMs from here. At this rate, GPT-5 may break the world.Want to learn more about the recent OpenAI drama? You can read our article on Sam Altman here or our other articles like this one.

Want to Learn Even More?

If you enjoyed this article, subscribe to our free newsletter where we share tips & tricks on how to use tech & AI to grow and optimize your business, career, and life.


Written by John Angelo Yap

Hi, I'm Angelo. I'm currently an undergraduate student studying Software Engineering. Now, you might be wondering, what is a computer science student doing writing for Gold Penguin? I took up studying computer science because it was practical and because I was good at it. But, if I had the chance, I'd be writing for a career. Building worlds and adjectivizing nouns for no other reason other than they sound good. And that's why I'm here.

Subscribe
Notify of
guest

0 Comments
Most Voted
Newest Oldest
Inline Feedbacks
View all comments