Latest Thoughts
-
š§ OpenAI Unveils Structured Outputs
OpenAI has introduced a groundbreaking feature that lets developers define a schema to receive responses from ChatGPT models using that schema. This enhancement moves beyond the traditional text responses, opening up dynamic ways to use the information across various systems.
Previously, prompt responses were limited to text answers, which restricted their application in feeding other systems. Now, you can define a schemaāfor instance, a data structure for a recipeārequiring the model to break down each step into an array or list and organize ingredients similarly.
This structured approach simplifies integrating the data directly into databases or crafting interfaces and formats for the response data, making developers’ lives easier and unlocking exciting new possibilities.
Iām looking forward to testing out these new structured outputs!
-
š§ Is Suno a Modern-Day Napster Saga?
It feels like the Napster days are back as the RIAA gears up to sue AI music companies Udio and Suno. Interestingly, these companies have countered the allegations by claiming fair use of copyrighted data.
āWe train our models on medium- and high-quality music found on the open internetā¦ Much of this indeed contains copyrighted materials, some of which are owned by major record labels,” says Shulman. He firmly believes that “Learning is not infringing. It never has been, and it is not now.”
The RIAA has substantial reasons to pursue this case, especially as signs emerge of a slowdown in music streaming (https://www.bloomberg.com/news/newsletters/2024-08-04/is-the-music-industry-slowdown-a-crisis-or-a-blip), with some analysts suggesting the industry may have peaked. This could prompt the organization to aggressively seek new revenue streams through litigation.
AI music startup Suno claims training model on copyrighted music is āfair useā | TechCrunchFollowing theĀ recent lawsuitĀ filed by the Recording Industry Association of America (RIAA) against music generation startups Udio and Suno, Suno admitted Suno admitted that it trained its AI model using copyrighted music but doing so was legal under the fair-use doctrine. -
š§ AI Companies are Forcing Websites to Play Whack-a-Bot
404 Media has an insightful piece on the complexities of correctly blocking botsāa topic perfectly aligned with my recent newsletter on AI and the robots.txt file.
Anthropic, the creators of Claude, are actively indexing content on the public web. However, the names of the bots and crawlers they employ seem to be in flux or are changing frequently. This makes it difficult, if not impossible, to tell AI tools not to consume your content.
I believe this isnāt necessarily a nefarious action, particularly from a company that emphasizes making AI safe. However, this constant name-changing makes it challenging to ensure youāre blocking the intended bots.
In an ideal scenario, robots.txt would allow for a whitelist approach, enabling us to specify who can access our content and compelling companies to maintain consistent bot names. Alternatively, it might be time to adopt Redditās approach and block everything, sending both search engines and AI bots a 404 error page.
-
š§ The First Taste of Apple Intelligence is Here
Apple has forked its betas, with iOS and iPadOS 18 releases continuing, likely targeting a general release in September with brand new iPhones.
Developers on Macs, iPhones, and iPads that meet the Apple Intelligence requirements (Apple Intelligence Requirements) can now download iOS and iPadOS 18.1, as well as macOS 15.1, giving a short peek at some of the early features.
The current suggestion is that Apple Intelligence features will ship in October. I still expect we will see some new features not announced at WWDC specific to the new round of devices.
I have the features enabled on my laptop, but sadly Iām one generation behind on my phone, so Iāll have to wait and see. For now, 9to5Mac has a rundown of the features available in the beta.
-
š§š¾āš» Having Problems with Drupal, Lando, and Pantheon?
I just spent forever trying to pull a new Drupal 10 website repo from Pantheon and get it running locally with Lando. Brutal. Iām documenting my path in hopes it helps some person some day, which started with this Reddit post: Problem getting Lando set up with Pantheon.
Steps to Get Your Drupal Site Running Locally with Lando and Pantheon:
- Clone the Site from Pantheon:
git clone <pantheon-repo-url>
This command clones the repository from Pantheon to your local machine. - Initialize Lando:
lando init- Do not select Pantheon as your environment.
- Select the working path where you pulled the site.
- For the second option, select Pantheon.
- Start Lando:
lando start
This command starts the Lando environment for your site. - Pull the Database and Files:
lando pull
This command pulls the database and files from Pantheon. (This step WILL fail due to database access issues.) - Destroy the Current Environment:
lando destroy
This command destroys the current Lando environment, which can help resolve issues with database access. - Rebuild the Environment:
lando start
This command starts the Lando environment again, which should rebuild the database correctly. - Pull the Database and Files Again:
lando pull
This command pulls the database and files from Pantheon again, which should now be successful. - Clear All Caches:
lando drush cr
This command clears all caches, which can help resolve errors and stability issues. If you haven’t you may need to install Drush first with this commandlando composer require drush/drush
Now, if youāre lucky, the gods have smiled upon you and you have a running site. Hoping to see this updated in the Pantheon docs or the Lando recipe updated sooner than later.
- Clone the Site from Pantheon:
-
š§ Zuckerbergās Thoughts on Open Source AI
Metaās release of the largest open-source AI model, Llama 3.1, with 405 billion parameters, continues to position the company as a leader in open AI. This move is helping to build a growing ecosystem around these models thatās simply not possible with commercial models from OpenAI, Claude, and others.
I have found the freedom of running the smaller Llama 2 model on my local environment to be liberating, providing a low-stakes way to experiment with new data approaches. The substantial investment in building these models is sure to pay off.
Of course, Llama is not the only open-source AI model available. Hugging Face is filled with AI models of all sorts, and the French AI developer Mistral continues to release more powerful models. Notably, Mistral has released some of its newer models under non-production or research licenses that forbid commercial use and deployment (Mistral License).
Meta also requires a license, which is thankfully simple and easy to read. As you can see (Meta Llama 3 License), it offers flexibility but requires applications using it to state āBuilt with Meta Llama 3ā and to prefix model names with “llama” before sharing them. Open-source purists are sure to disagree with these stipulations, seeing them as a departure from the true spirit of open-source freedom.
This may be a small price to pay for access to frontier-level models, but I wish more companies would embrace the idea of exploration and open source, providing last-generation releases to the developer community.
Zuckerbergās blog post on Metaās place in AI is a worthwhile read. I plan to dive into it in a future newsletter, so subscribe and stay tuned.
-
š§ OpenAI Just Released Search!
I’m surprised it took so long. After all, OpenAIās ChatGPT powers Microsoftās Bing search, so in some ways, the company has been in the search game from nearly the start.
What’s interesting is that OpenAIās approach is less like Bing and Google’s AI Overviews and more like Perplexity AIāmy favorite new search tool in years. This is a good thing, changing our relationship with search from a list of results that may hold the answer to our questions, to actual responses that you can drill into with additional questions.
For access you need to join a waitlist, and I’m on it, so I canāt kick the tires just yet. OpenAI expects to integrate search into ChatGPT in the long term rather than maintaining them as separate products.
This means the competition in search is heating up for Googleāand so far, their attempts to add AI to search have been lacking.
-
š§ Why Google is no longer limiting third party cookies in Chrome
The beauty of digital marketing lies in its ability to be highly targeted. If you run ads on social media or through Googleās vast ad platform, you know how amazing it is to target based on criteria like gender, education level, income, and location.
These targeted ads perform better than untargeted ones, allowing you to reach customers at lower acquisition costs. However, this requires platforms to gather as much information about you as possible and track your activity across the internet and apps to send you these highly targeted ads.
Several years ago, Apple introduced features to its platforms that prevented this type of tracking through its apps and browser as part of its privacy stance. This move was quickly followed by other browsers like Firefox. Google also promised to do the same but has repeatedly delayed implementing this change (Google halts its 4-plus-year plan to turn off tracking cookies by default in Chrome).
So why did Google get cold feet? In 2022, Meta (then Facebook) saw a $250 billion drop in valuation due to Appleās new rules (How Apple helped create Facebookās $250 billion ad crisis). Despite Google’s size, a significant portion of its revenue comes from ads. Blocking third-party cookies in the worldās most-used web browser would have a seismic impact on its business. Google’s hope was to block third-party cookies in Chrome while replacing them with a worse tracking method called FloC (Googleās FLoC Is a Terrible Idea), but as that path became unattainable, it decided to backtrack on its previous plan.
This leaves us with a new ad campaign launched by Apple, reminding folks not to use Chrome unless they want their browser to track, watch, and report everything they do.
So, what browser are you using?
-
š§ How Will Global Copyright Rules Shape AI Development?
I’ve often wondered how differing copyright rules and protections across the globe might impact AI. The Financial Times has an interesting write-up on how Japan is skirting international copyright rules to become a leading destination for AI startups.
This is an interesting counter to the EU, which has been a regulation hawk on AI, to the extent that both Apple and Meta have decided to delay the release of key AI features in their member countries.
In a world where, as Nvidia’s CEO Jensen Huang aptly put it: “While some worry that AI may take their jobs, someone who’s expert with AI will,” this sentiment applies equally to businesses. Those businesses that learn to harness AI will create a competitive advantage that could make the difference between thriving and merely surviving in the coming years.
-
š§ CrowdStrike Reminds Us That Dependency Management is a Major Attack Vector
I’m sure you’re aware of one of the biggest computer shutdowns ever, which has grounded over 2,000 flights worldwide, shuttered hospitals, retail stores, and who knows what else. All because of a faulty automatic update from CrowdStrikeāsoftware ironically meant to protect companiesāthat’s used by a majority of Fortune 500s.
My take is that this issue has been brewing for some timeāautomatic updates and increasing failures in dependency management.
For the non-developers in the room, most modern software is built on tons of dependencies from a combination of open-source and closed-source repositories. When you set up a new project or do an update, a dependency management tool downloads the code from an external source so your application can use it.
Our software has tons of dependencies, much of this to allow us, as developers, to avoid rebuilding logic that’s already been built. Think database connections, complex math operations, image editing tools, charts and more.
In 2016, a developer of a package used by tons of software rage-quit over issues with his package’s name, breaking the Internet and bringing down tons of applications around the world.
This has become an increasingly common attack vector where squatters buy up abandoned packages only to add code that can be used as a back door, or worse, hackers become contributors to popular packages with the goal of injecting malicious code into applications.
While all eyes point to a mistake by CrowdStrike, it serves as a reminder that our software has become so complex that auto-updates of dependencies, like security software or operating system updates on production platforms without testing, remain a huge door for vulnerabilities.
IT managers should not let software publish to mission-critical systems, even from their most trusted vendors, without testing and ensuring a ready rollback procedure to get systems quickly up in the case of a failure.
-
š§ OpenAI Introduces ChatGPT 4o Mini
Not a ton of detail yet, but interesting to see OpenAI push into the realm of Small Language Modelsāespecially with a multimodal model that can easily understand audio, video, images, and text.
The round-trip latency cost of calling services on the Internet can make or break AI-powered hardwareājust look at the Humane pin.
Powerful small models that run locally can open the door to super-fast response times that happen locally, and with fewer privacy concerns.
Of course, details are super slim, but if OpenAI plans to license ChatGPT 4o Mini to hardware makers it could open the door to tons of exciting new products that actually work.
-
š§ Is Meta’s Multi-Token Prediction Model A Game-Changer?
Meta just released a multi-token prediction model that could cut down inference, the process where a model responds to a prompt, speed by 3x.
Existing LLMs work like autocomplete, predicting the next token or word in a sequence. This novel approach looks to predict 2-4 words in a sequence all at once, allowing for faster response times.
Meta released the model under a research license using Hugging Face, continuing to solidify its place as the open-source AI leader. I keep saying it, but who would have thought Meta would be blazing new paths?
-
š§ Should Governments Have Backdoor Access to Encrypted Devices?
Should law enforcement have a backdoor to our phones or digital worlds? This question seems to pop up every few years, often following a tragic eventāin this case, the phone of an attempted assassin.
In my opinion, end-to-end encryption makes a backdoor impossible, as any method to design one would create a loophole that hackers could exploit to access information. For example, consider the TSA’s requirement to have access to all luggage. The idea was that only TSA officials would have a backdoor key to open luggage, keeping it safe. However, like many well-intentioned plans, that key is now easily available on the internet, meaning no luggage is actually safe.
So far, there’s no news on whether this is an iPhone or an Android device, but letās see. Where do you stand? Should end-to-end encryption be allowed without exceptions? Should governments have the right to require backdoors into encryption?
-
š§ OpenAI Reveals the 5 Steps to AGI
Bloomberg reported on OpenAIās five levels AI must meet to achieve Artificial General Intelligence (AGI). Currently, OpenAI says ChatGPT 4o is at level 2.
- Level 1: Chatbots, natural language
- Level 2: Reasoners, can apply logic and solve problems at a human level
- Level 3: Agents, can perform additional actions
- Level 4: Innovators, can make new inventions
- Level 5: Can do the work of an entire organization
Each ChatGPT release pushes us closer to the next level. ChatGPT 3.5 is level 1, providing a chatbot that could understand and respond to natural human language. ChatGPT 4o, a multimodal model, is level 2, able to reason and solve problems, surpassing human performance on many tests.
The next milestone, potentially with ChatGPT 5.0 in 2025, will be level 3, envisioning an AI acting as an independent agent, able to read emails, understand content, and perform tasks without direct human prompting.
Level 4 aims for innovation, where AI not only learns but also creates new ideas and solutions.
Level 5 represents the ultimate goal, where AI can manage an entire organization, handling multiple tasks and actions autonomously. Think of Rosy from “The Jetsons” managing a homeāscheduling, cleaning, cooking, and maintaining order without human intervention.
Sam Altman, CEO of OpenAI, suggests that we might achieve AGI before the end of the decade. Who knows what ChatGPT 7 or 8 might bring? All before 2030.
-
š§ Creating Virtual Worlds with Text-to-3D
Meta’s announcement of text-to-3D is an obvious next step in the company’s mixed reality goals. For a few months, I’ve used an AI tool named Meshy that allows you to create 3D assets and export them as USDZ files. Once in this format, it’s easy to grab and manipulate these 3D assets on Apple Vision Pro or Meta Quest as real-world AR objects.
This announcement is just the start of some cool social media capabilities in Meta’s Horizon environment, allowing you to create virtually anything from photos or text and transform them into 3D objects you can wear.
Imagine uploading a favorite dress, pair of shoes, or a weapon from a cartoon and instantly converting that object into a 3D object you can mount in a 3D environment or place on a virtual avatar. It’s early days, but bit by bit, the groundwork is being laid to make all of this possible.
Meta shows off ā3D Genā AI tool that creates textured models faster than ever.Metaās AI research team has a new system to create or retexture 3D objects based on a text prompt. It combinesĀ text-to-3D and text-to-texture generation models to go beyond AI-generated emoji or still images, Their paper (pdf) claims 3D Genās output is ā3Ć to 60Ć fasterā and preferred by professional artists in comparison to alternatives. -
š§ Would You Sell Your Digital Self?
On his death, Wendyās founder Dave Thomas gave the company the right to digitally recreate him for ads. In 2002, that seemed crazy, but today, de-aging tech and body scans have reached a point where reality and fiction have blended.
ElevenLabs recently announced a deal to buy the voice rights of deceased celebrities from their estates, allowing companies to pay to use their voices for various projects, like audiobooks.
With AI recreations of Marilyn Monroe and hologram 2Pac at concerts, itās possible to imagine artists who, through their estates, continue to create music, act, and write books through digital avatars long after their death.
Should this be normalized? Would you sell the rights to your digital self to help provide for your family after you leave?
/cdn.vox-cdn.com/uploads/chorus_asset/file/23982723/acastro_STK004_01.jpg)
