Why Llama 3.1 405B Is So Much Better Than GPT-4o And Claude 3.5 Sonnet Here The Result

the AI news in the past 7 days has been insane  with so much happening in the world of AI   in this video  were diving into some of the latest AI developments from major players like Llama 3.1 405B  GPT-4o and Claude 3.5 Sonnet   Llama 3.1 405B is the first open-source model that performs on par with leading proprietary AI models in general knowledge  steerability  math  tool use  and multilingual translation  among other capabilities.   Meta announced the launch of Llama 3.1  which is the largest open-source AI model to date  and has surpassed OpenAIs GPT-4o and Anthropics Claude 3.5 Sonnet in multiple benchmark tests!   In this step-by-step guide  we will cover what Llama 3.1 405B is  how to use Llama 3.1 405B locally  and why Llama 3.1 405B is so much better than GPT-4o and Claude 3.5 Sonnet.   I highly recommend you watch this video to the end is a game changer in your chatbot that will realize the power of Llama 3.1 405B!   If you like this topic and you want to support me:   Clap my article 50 times; that will really help me out.U+1F44FFollow me on Medium and subscribe to get my latest articleU+1FAF6Follow me on my YouTube channelMore info on my discordLlama 3.1 405B is Metas largest model  trained with over 15 trillion tokens. For this  Meta optimized the entire training stack and trained it on more than 16 000 H100 GPUs  making it the first Llama model trained at this scale.   According to Meta  this version of the original model (Llama 1 and Llama 2) has 128K context length  improved reasoning and coding capabilities. Meta has also upgraded both multilingual 8B and 70B models.   Key Features of Llama 3.1 40 5B:Llama 3.1 comes with a host of features and capabilities that appeal to The users  such as:   RAG & tool use  Meta states that you can use Llama system components to extend the model using zero-shot tool use and build agentic behaviors with RAG.   Multi-lingual  Llama 3 naturally supports multilingual processing. The pre-training data includes about 50% multilingual tokens and can process and understand multiple languages.   Programming and Reasoning  Llama 3 has powerful programming capabilities  generating high-quality code with a strong understanding of syntax and logic. It can create complex code structures and perform well in various programming tasks. Llama 3 excels in logical reasoning  problem-solving  analysis  and inference. It handles complex logical tasks and solves intricate problems effectively.   Multimodal Models  Multimodal models have been developed that support image recognition  video recognition  and speech understanding capabilities  but these models are still under development and have not yet been widely released.   Benchmark ResultsMeta compared the Llama 3.1 405B model with models such as GPT-4  GPT-4o  and Claude 3.5 sonnet. The results showed that Llama 3.1 performed better than GPT-4o and Claude 3.5 sonnet on test data sets such as mathematical reasoning  complex reasoning  and Multilingual support and its long text processing ability is also excellent  receiving 95.2 points in zero scrolls quality.   it falls short compared to Claude 3.5 sonnet in tool utilization ability (BFCL  Nexus).   Although the performance on test data sets such as Multi-task Language Understanding  Human Eval  and MATH is slightly inferior to the closed-source model  the score difference is not large.   In addition  manual evaluation results show that the output performance of the Llama 3.1 405B model is comparable to GPT-4 and Claude 3.5 Sonnet  and slightly inferior to GPT-4o.   Just looking at this benchmark score  it seems to be quite promising.The benchmark results show that llama 3.1 405B is an excellent language model with strong language modeling capabilities  mathematical reasoning capabilities  complex reasoning and long text processing capabilities. however  there is still room for improvement in tool utilization capabilities and multilingual support.   Now that Ive introduced the benchmark scores  Im going to try using them and see how they perform.   How to Use Llama 3.1 40 5B Locally?Ollama is the fastest way to get up and running With local language models We recommend trying Llama 3.1 8b  which is impressive for its size and will perform well on most hardware.   Download Ollama here (it should walk you through the rest of these steps)Open a terminal and run ollama run llama3.1-8bGroq is now hosting the Llama 3.1 models  including the 70B and 8B models. Earlier  they offered the largest 405B model  but it has been temporarily removed due to high traffic and server issues. Meanwhile  the Llama 3.1 70B and 8B models are still available and can generate responses at an impressive speed of 250 tokens per second.   head to groq.com and sign in with a free account. Next  choose the llama 3.1 model from the drop-down menu.   Llama 3.1 405B vs Claude 3.5 Sonnet vslets compare Claude 3.5 Sonnet and GPT-4o and verify how much performance Llama 3.1 405B has.   This time  we will verify the following items.   Coding   I went to the LeetCode platform and selected a medium-level problem related to sorting algorithms. The problem  titled Top K Frequent Elements problem involves finding the K most frequent elements in a given dataset to test out.   Given an integer array nums and an integer k  return the k most frequent elements. You may return the answer in any order. Example 1 Input: nums = [1 1 1 2 2 3]  k = 2 Output: [1 2] or [2 1] Example 2 Input: nums = [1]  k = 1 Output: [1] Constraints 1 <= nums.length <= 105 -104 <= nums[i] <= 104Llama 3.1 405B   ```python import heapq from collections import Counter def topKFrequent(nums  k):  Returns the k most frequent elements in the given array. Args: nums (list): The input array of integers. k (int): The number of most frequent elements to return. Returns: list: The k most frequent elements in any order.  # Count the frequency of each element count = Counter(nums) # Use a heap to find the k most frequent elements return heapq.nlargest(k  count.keys()  key=count.get) ''' Here's an explanation of the code: 1. We use the `Counter` class from the `collections` module to count the frequency of each element in the input array `nums`. This gives us a dictionary `count` where the keys are the unique elements in `nums` and the values are their frequencies. 2. We use the `heapq.nlargest` function to find the k most frequent elements. We pass `k` as the first argument  `count.keys()` as the second argument (which gives us an iterable of the unique elements in `nums`)  and `key=count.get` as the third argument (which tells `heapq.nlargest` to use the frequency of each element as the key for comparison). 3. The `heapq.nlargest` function returns a list of the k most frequent elements in any order. Example use cases: ```python print(topKFrequent([1 1 1 2 2 3]  2)) # [1  2] or [2  1] print(topKFrequent([1]  1)) # [1] ``` Note that this solution has a time complexity of O(n log k) where n is the length of the input array  because we use a heap to find the k most frequent elements. This is efficient for large inputs because the heap operations are logarithmic in the size of the heap.GPT-4o   To solve the problem of finding the k most frequent elements in an integer array  you can use a combination of a dictionary to count the occurrences and a heap to keep track of the top k elements. Heres a step-by-step approach: Count the Frequencies: Use a dictionary to count the frequency of each element in the array. Use a Heap for Top k Elements: Use a min-heap to keep track of the top k elements based on their frequencies. Pythons heapq library is useful for this purpose. Extract the Elements from the Heap: After processing the array  the heap will contain the k most frequent elements. Heres the implementation of the above logic: import heapq from collections import Counter def topKFrequent(nums  k): # Step 1: Count the frequency of each element count = Counter(nums) # Step 2: Use a heap to keep track of the top k elements heap = [] for num  freq in count.items(): heapq.heappush(heap  (freq  num)) if len(heap) > k: heapq.heappop(heap) # Step 3: Extract the elements from the heap return [num for freq  num in heap] # Example 1 nums1 = [1  1  1  2  2  3] k1 = 2 print(topKFrequent(nums1  k1)) # Output: [1  2] or [2  1] # Example 2 nums2 = [1] k2 = 1 print(topKFrequent(nums2  k2)) # Output: [1]Claude 3.5 Sonnet   from collections import Counter import heapq def topKFrequent(nums  k): # Count the frequency of each number count = Counter(nums) # Use a min heap to keep track of the k most frequent elements heap = [] for num  freq in count.items(): if len(heap) < k: heapq.heappush(heap  (freq  num)) elif freq > heap[0][0]: heapq.heappop(heap) heapq.heappush(heap  (freq  num)) # Extract the numbers from the heap return [num for freq  num in heap] # Test the function print(topKFrequent([1 1 1 2 2 3]  2)) # Expected output: [1  2] print(topKFrequent([1]  1)) # Expected output: [1]All three solutions were correct when I ran the code on Visual Studio code. However  Llama3s solution was the most concise because it directly used heapq.nlargest.   In contrast  GPT-4 and Claude 3.5 Sonnet managed the heap manually. Llama3's solution might be slightly more efficient and straightforward due to its direct use of heapq.nlargest.   Overall  each solution solves the problem correctly  but Llama3's approach stands out for its simplicity and direct use of Python's heap functions   MathematicsThen I compare and verify mathematical abilities  we will have participants answer probability questions  a key concept in data science.   Alice has 2 kids and one of them is a girl. What is the probability that the other child is also a girl? You can assume that there are an equal number of males and females in the world. A) 0.5 B) 0.25 C) 0.333 D) 0.75 Llama 3.1 405B   This is a classic problem! At first glance  it might seem like the probability is 50% (1/2)  since the other child could be either a boy or a girl. However  this is actually a subtle mistake. When we say one of the children is a girl  we're not specifying which child is the girl. This means that the girl could be either the first child or the second child. Let's analyze the possible scenarios: 1. The first child is a girl  and the second child is a boy (GB). 2. The first child is a girl  and the second child is a girl (GG). 3. The first child is a boy  and the second child is a girl (BG). Notice that we've already accounted for the fact that one of the children is a girl. Now  we want to find the probability that the other child is also a girl. Out of the three possible scenarios  only one (GG) has both children as girls. However  we need to consider that the girl could be either the first or the second child  so we have two possible ways to get the GG scenario: the first child is a girl and the second child is a girl  or the first child is a girl (but we don't know that yet) and the second child is a girl (which we do know). So  when we say one of the children is a girl  we're essentially combining the GB and BG scenarios  as well as the GG scenario. This means we have a total of 3 possible scenarios  and 1 out of those 3 has both children as girls (GG). However  since we have already established that one of the children is a girl  the sample space is reduced to the 3 scenarios mentioned earlier (GB  GG  and BG)  with each scenario being equally likely. And out of those 3 scenarios  1/3 of them have the other child also being a girl. Therefore  the probability that the other child is also a girl is 1/3  or approximately 33.33%. It's worth noting that this problem is often counterintuitive  and many people initially think the probability is 50%. However  by carefully analyzing the possible scenarios and considering the reduced sample space  we can see that the correct probability is indeed 1/3.GPT-4o   to solve this problem  we need to analyze the possible scenarios and apply basic probability principles. Given that Alice has two children and we know one of them is a girl  we want to determine the probability that the other child is also a girl. List all possible combinations of two children: Girl  Girl (GG) Girl  Boy (GB) Boy  Girl (BG) Boy  Boy (BB) Exclude the combinations that do not meet the given condition (one of the children is a girl): The combination Boy  Boy (BB) does not meet the condition because neither child is a girl. So  we are left with: Girl  Girl (GG) Girl  Boy (GB) Boy  Girl (BG) Determine the probability: We now have three equally likely combinations: GG  GB  and BG. In two of these combinations (GB and BG)  the other child is a boy. In one of these combinations (GG)  the other child is a girl. Thus  the probability that the other child is also a girl is: Probability= Number of favorable outcomes /Total possible outcomes = 1/3  Therefore  the probability that the other child is also a girl is 1/3 or approximately 33.33%.Claude 3.5 Sonnet   Let's approach this step-by-step: First  let's consider all possible combinations of two children: Boy-Boy (BB) Boy-Girl (BG) Girl-Boy (GB) Girl-Girl (GG) We're told that Alice has at least one girl. This eliminates the BB possibility. So we're left with three equally likely possibilities: Boy-Girl (BG) Girl-Boy (GB) Girl-Girl (GG) Out of these three possibilities  only one (GG) satisfies the condition that the other child is also a girl. Therefore  the probability is: 1 (favorable outcome) / 3 (total possible outcomes) = 1/3 Thus  the probability that Alice's other child is also a girl is 1/3 or about 33.33%.All three models correctly identify that the probability is 1/3. Llama 3.1 gives the most detailed and thorough explanation  which is helpful for readers who might not know the problem well.   GPT-4 offers a clear and easy-to-understand explanation but lacks some details that could help readers new to the concept.   Claude 3.5 Sonnet strikes a balance between clarity and detail  giving a straightforward answer   Conclusion:The release of Llama 3.1  especially the strong performance of the 405B parameter model and the introduction of knowledge distillation  has greatly improved open-source language models. For the first time in recent years  the performance of open-source language models is very close to that of closed-source business models. This shows that in the future  Metas Llama series models will likely remain the top choice for developing open-source language models   U+1F9D9U+2642 I am AI application expert! If you want to collaborate on a project  drop an inquiry here or Book a 1-On-1 Consulting Call With M.   U+1F4DAFeel free to check out my other articles:   Why OpenHermes-2.5 Is So Much Better Than GPT-4 And LLama2 13B  Here The Resultthe AI news in the past 7 days has been insane with so much happening in the world of AIpub.towardsai.net   Why OPENCHAT Model is So Much Better Than ChATGPT?Hugging Face recently announced their new open-source Large language model  OpenChat  which is a fine-tuned version ofai.plainenglish.io   Why Command R+ is Much Better Than Mistral Large and Offers the Same Level of Performance asWhen I look at my Twitter timeline these past few days  I see quite a few tweets about  Command R+  its a Largepub.towardsai.net


What is Claude AI  and How Does it Differ From ChatGPT?

Claude AI and ChatGPT are both powerful and popular generative AI models revolutionizing various aspects of our lives. Here  let us learn more about Claude AI and its benefits   Ever since the launch of ChatGPT  many other companies have also joined the race to bring excellent generative AI models into the world that not only help users create realistic content but are also safe to use  and free from bias. While Open AIs ChatGPT and Googles Bard  now Gemini  get most of the limelight  Claude AI stands out for its impressive features and being the most reliable and ethical Large Language Model.   In this article  we will learn more about what Claude AI is and what are its unique features. We will also discuss how it differs from the most popular generative AI tool ChatGPT.   Claude AI is developed by Anthropic  an AI startup company backed by Google and Amazon  and is dedicated to developing safe and beneficial AI. Claude AI is an LLM based on the powerful transformer architecture and like OpenAIs ChatGPT  it can generate text  translate languages  as well as write different kinds of compelling content. It can interact with users like a normal AI chatbot; however  it also boasts some unique features that make it different from others.   1. Larger Context Window   One of the Claude AIs biggest capabilities is that it can process huge chunks of text as compared to ChatGPT. While ChatGPT struggles to process and keep track of information in long conversations  Claudes context window is huge (spanning up to 150 pages)  which helps users to do more coherent and consistent conversations  especially when it comes to long documents.   2. Dedicated to safety and security   It is a well-known fact that Anthropic prioritizes responsible AI development the most  and it is clearly seen in Claudes design. This generative AI model is trained on a carefully curated dataset thus it minimizes biases and factual errors to a large extent. On top of that  Claude also undergoes rigorous safety checks to prevent the generation of harmful and misleading content.   3. Emphasizes Explainability   While many of the AI and LLMs currently operate as black boxes  Claude offers a high level of explainability surpassing other models. This means it can explain the reasoning and decision-making process behind all of its responses. Therefore  it helps users to use this model confidently and they can be assured about the credibility of the information provided.   Claude FamilyClaude AI comes in a family of 3 generative AI models. Users can choose from these three models based on their power requirements and budget.   1. Haiku: It is the most budget-friendly option and offers fast response times. This can be perfect for simple tasks that require short context. This is yet to be launched but users can expect it to be highly cost-effective and cheaper as compared to other models.   2. Sonnet: This is a free-tier model and serves as an excellent starting point by offering a balance between cost and features. It can effectively handle tasks like writing different creative text formats and answering questions  just like Open AIs ChatGPT.   3. Opus: This is the most powerful generative AI model by Claude AI; however  users require a premium subscription to use this AI Chatbot. It can perform complex tasks easily that require a large context window. So  if you are looking for a generative AI that can do research  summarize lengthy documents  or help with consistent lengthy conversations  then this model will be the best option.   ChatGPT vs. Claude AI: How do they differ?Claude AI and OpenAIs ChatGPT both are very powerful LLM models. But they are designed for various purposes. Lets compare.   Strengths:    Claude: It is great in performing tasks requiring long-term context as discussed above. They can maintain consistency throughout their response in extended conversations. Also  their explainability makes them more attractive.    ChatGPT: They are designed for multiple tasks and can help users generate texts  codes  as well as images through its image generation generative AI tool Dall-E. It also has internet access to retrieve information in real time. Also  it can effectively process voice prompts in its paid tiers.   Weaknesses    Claude: Claude offers a free tier which is very powerful. But its paid model lacks a few advanced features like data analysis and image understanding that are offered by ChatGPT+    ChatGPT: the free version uses GPT 3.5 which is less powerful than Claudes base model. For all advanced features  users need to subscribe to their paid versions.   Become an AI Prompt EngineerThe use and development of such generative AI models is on the rise  and it has opened the door for fascinating career paths like AI Prompt Engineers. These AI professionals specialize in crafting instructions (prompts) that guide LLMs to give desired outputs.   According to Glassdoor  the annual average salary of AI Prompt Engineers in the US is $128 081 per annum. To be an efficient AI prompt engineer  you need to have a strong understanding of AI  and its components like Natural Language Processing and the specific capabilities of the LLM model youre working with.   To excel in this career path  it is also recommended to pursue an AI certification. Though they are not mandatory  earning a generative AI certification can help you demonstrate your skills and expertise in using AI models. This will enhance your chances of getting hired faster and with a higher salary. So  enroll now.   ConclusionClaude AI is a powerful LLM that is more focused on providing responses that are accurate  ethical  and correct. They excel in processing long-form content and also offer clear explainability to their users. Though it may not offer as many features as its competitor ChatGPT  it specializes in performing specific tasks and responsible generative AI model development. As they continue to evolve  the future of LLMs looks bright with exciting possibilities across various domains. So  which one to choose- Claude AI or ChatGPT? Well  it all depends on your specific needs and priorities.


How to Use Functional Programming Features in Python?

Functional programming (FP) is a programming paradigm that emphasizes the use of pure functions for computation and data processing.   Although Python is not a purely functional programming language  it offers many features that support functional programming  including anonymous functions (lambda)  higher-order functions  immutable data structures  and numerous functional programming libraries and modules.   1.Pure Functions and Side EffectsPure functions are one of the core concepts in functional programming. A pure function always returns the same output given the same input and has no side effects. Side effects refer to a function modifying external state or interacting with external systems (such as modifying global variables or performing I/O operations) aside from returning a value.   1.1 Example of Pure Functions   def add(x  y): return x + y print(add(2  3)) # Output: 5The add function is a pure function because it always produces the same output given the same input and does not modify any external state.   1.2 Example of Side Effects   global_var = 0 def impure_add(x  y): global global_var global_var += x + y return global_var print(impure_add(2  3)) # Output: 5 print(global_var) # Output: 5The impure_add function is not a pure function because it modifies the external variable global_var.   2.Higher-Order FunctionsHigher-order functions are functions that take other functions as parameters or return functions as their results. Python supports higher-order functions  making functional programming more convenient.   2.1 Taking Functions as Parameters   A common example of a higher-order function is map  which takes a function and an iterable as parameters and returns a new iterable with the function applied to each element.   def square(x): return x * x numbers = [1  2  3  4  5] squared_numbers = list(map(square  numbers)) print(squared_numbers) # Output: [1  4  9  16  25]Here  make_multiplier returns a new function  which is a closure that captures the parameter n.   3. Anonymous Functions (Lambda Functions)Anonymous functions are functions without a name  often used in places where a short function is needed. Python creates anonymous functions using the lambda keyword.   3.1 Simple Example   f = lambda x  y: x + y print(f(2  3)) # Output: 5The expression lambda x  y: x + y defines an anonymous function that takes two parameters  x and y  and returns their sum.   3.2 Using Anonymous Functions in Higher-Order Functions   Anonymous functions are often used in higher-order functions such as map  filter  and sorted.   numbers = [1  2  3  4  5] squared_numbers = list(map(lambda x: x * x  numbers)) print(squared_numbers) # Output: [1  4  9  16  25]4.Immutable Data StructuresImmutable data structures are data structures that cannot be modified once they are created. Immutability is an important concept in functional programming as it makes data more secure and avoids side effects. Pythons immutable data structures include strings  tuples  and frozenset.   4.1 Example of Immutable Data Structures   Here are some examples of immutable data structures in Python:   tuple1 = (1  2  3) # tuple1[0] = 10 # This will raise an error because tuples are immutable frozenset1 = frozenset([1  2  3]) # frozenset1.add(4) # This will raise an error because frozensets are immutableTuples and frozensets cannot be changed once they are created  providing data immutability.   5.Common Functional Programming ToolsPythons libraries such as functools  itertools  and toolz provide many tools and functions for functional programming.   5.1 functools Module   functools offers several tools for higher-order functions and function operations.   5.1.1 reduce Function   The reduce function reduces an iterable to a single value by repeatedly applying a specified function to the first two elements of the iterable.   from functools import reduce numbers = [1  2  3  4  5] sum_numbers = reduce(lambda x  y: x + y  numbers) print(sum_numbers) # Output: 155.1.2 partial Function   The partial function is used for partial application of a function  meaning you can fix a few parameters of the function and generate a new function.   from functools import partial def power(base  exponent): return base ** exponent square = partial(power  exponent=2) cube = partial(power  exponent=3) print(square(4)) # Output: 16 print(cube(4)) # Output: 645.2 itertools Module   The itertools module provides many tools for iterating and combining data.   5.2.1 chain Function   The chain function connects multiple iterators to form a single iterator.   from itertools import chain numbers1 = [1  2  3] numbers2 = [4  5  6] combined = list(chain(numbers1  numbers2)) print(combined) # Output: [1  2  3  4  5  6]5.2.2 combinations Function   The combinations function returns all possible combinations of elements from the input iterable without repeating elements.   from itertools import combinations items = ['a'  'b'  'c'] combo = list(combinations(items  2)) print(combo) # Output: [('a'  'b')  ('a'  'c')  ('b'  'c')]5.3 toolz Library   toolz is an external library that provides more functional programming tools  such as curry and compose.   5.3.1 curry Function   The curry function is used for partial application of functions and is more flexible than functools.partial.   from toolz import curry @curry def add(x  y): return x + y add_five = add(5) print(add_five(10)) # Output: 155.3.2 compose Function   The compose function is used for function composition  allowing you to combine multiple functions into a single function. The functions are executed in right-to-left order.   from toolz import compose def double(x): return x * 2 def increment(x): return x + 1 composed_func = compose(double  increment) print(composed_func(3)) # Output: 8 (first increment(3) to get 4  then double(4) to get 8)6.Pros and Cons of Functional Programming6.1 Pros   Testability and Debuggability: Pure functions have no side effects  making them easy to test and debug.   Concurrency: Since there is no shared state  pure functional code is naturally suited for concurrent execution.   Predictability: Pure functions produce the same output for the same input  making the codes behavior more predictable.   6.2 Cons   Performance Overhead: Due to immutability and the use of recursion  there may be performance overhead.   Learning Curve: For programmers accustomed to imperative programming  understanding and applying functional programming can take time.   7.Functional Programming Application ExamplesFunctional programming is widely used in areas such as data processing  stream computing  and parallel computing. For instance  in data analysis  we often use functions like map  filter  and reduce to handle data sets.   7.1 Data Processing Example   Lets say we have a list of student grades  and we want to calculate the average grade for all students:   from functools import reduce students = [ {name: Alice  score: 88}  {name: Bob  score: 72}  {name: Charlie  score: 95}  {name: David  score: 85} ] average_score = reduce(lambda acc  student: acc + student['score']  students  0) / len(students) print(fAverage Score: {average_score}) # Output: Average Score: 85.0In this example  we used the reduce function to calculate the total score of all students and then divided it by the number of students to get the average score.   Pythons functional programming features provide powerful tools for writing concise  clear  and maintainable code.   Although Python is not a purely functional language  its functional programming features are sufficient for most application scenarios.   Mastering these features can help programmers write more reliable and maintainable code while improving code concurrency and testability.   By understanding and applying these concepts  developers can fully leverage the advantages of functional programming in Python  resulting in more efficient code.


How to Build With the Chromes Latest Built-in AI

Gemini Nano  built-in AI by Google is picking up steam lately. Google first announced built-in AI in this years I/O event. The model was subsequently launched in the newest Canary release and Dev channel.   The current default for building AI features for the web is server-side solutions. OpenAI and Anthropic are among the main players dominating the market. Other key players like Google seemed lagging behind. But it is changing now.   My first impression of Gemini Nano is finesse.   Local  private  and offline models are the future. We already have some tools that provide this to a certain extent  like LM Studio and Ollama. But ordinary users dont bother downloading models to run things locally. Thats where built-in AI comes in.   You can bring top-notch LLM capabilities to your users without compromising their privacy  with no middleman involved  and you can deliver a snappy user experience because you are eliminating network round trips.   In some cases  you can build offline first products where your users can access built-in AI even when they are not connected to the internet.   Setting up Gemini NanoYou need at least Windows 10 or MacOS 13  integrated GPU  and 22GB of storage (but the model doesnt take that much space  its just to make sure you have enough storage margin).   Gemini Nano is still in early preview. Therefore  you need to download the Chrome Dev channel or Canary channel and confirm that your version is equal to or newer than 128.0.6545.0.   After installing Canary or dev channel  go to chrome://flags/#optimization-guide-on-device-model and select Enabled BypassPerfRequirement.   Then go to chrome://flags/#prompt-api-for-gemini-nano and select Enabled.   After that  restart Chrome.   To test that you have configured everything correctly  open DevTools and enter this in the console:   await window.ai.canCreateTextSession();If you got readily then you are all set.   Now we are ready to hack.   Lets cook something...As a developer  I think built-in AI will help my users quickly skim information on my website.   So  I am going to implement Gemini Nano on my site for AI jobs to help job seekers quickly get some info from job descriptions such as salary details  location restrictions  and minimum qualifications  all without needing to read through the 300-word description. This can help improve the productivity of job seekers.   I assume similar use cases for many other sites  such as recipe websites  product reviews  tutorials  etc. It is also for content creation-related tasks such as proofreading  rephrasing  and grammar correction.   I am using Next.js for the site  so the following code will be on Next.js.   The basic idea is to show a chat widget on the corner for users to chat with the job description. To do that  we need to pass the job description to Gemini Nano and direct user questions to it to answer based on the job description.   Next.js components are RSC by default  so we need to declare a client component.   First  we need a client component to access the built-in AI because the Gemini Nano is on the users device. To mark a component as a client component  we need to use a directive called use client.   To do so  put this at the beginning of the component  above any imports or other code:   'use client';After that  we need to make sure the user device can run Gemini Nano and is ready to receive prompts  otherwise  they cant use the chat feature and we shouldnt display the chat widget for them.   Now lets write the code to receive questions from users and send them to built-in AI:   We are using the react-markdown library to render results from Gemini Nano as markdown.   To put it all together:   ConclusionWhile all this is crazy  the Chrome team recently shipped session persistence. The best part about this is you dont need to keep track of the entire conversation history as we do with OpenAIs chat completion endpoint. Chrome will preserve the context for you. If you need to destroy the context and start an entirely new conversation  you can do:   session.destroy();Its pretty awesome to see where built-in AI is headed  and excited to see what other developers will build with this.   Thats it for this post. Please let me know in the comments below if you have any questions. I would love to help.   Thank you for reading. I build my products in public on Twitter and here. Please make sure to follow me if you are interested in building stuff together.


The Problem with Prompting and What it Really is

This post is an older iteration of our weekly newsletter High Learning Rate. Follow it to get more of such posts 1 month before!   Lets discuss the current problem with promptingmore precisely  advanced prompting and those who call themselves prompt engineers.   Despite all the hype around advanced prompting techniques  its really just about telling the model what you want in plain language.   Its all about good communication.   Giving directions  be clear  be concise  and maybe add some examples if youre feeling generous.   Heres our short opinion piece on prompting. Well start by covering the basics and sprinkle in some so-called advanced techniques  which are mostly just common sense wrapped in fancy terminology.   1. Understanding PromptsA prompt is the input or instruction given to a model to generate text. Designing an effective prompt can significantly enhance the quality and relevance of the models responses. But theres nothing too complex. Its just good communication  tell what you want clearly and concisely  and add examples if you can.   In their simplest form  prompts can be questions or instructions  e.g.  asking a model to complete a sentence:   Prompt: Complete this sentence: The sky isOutput: The sky is blue during the day and dark at night.While this is elementary  more context or detailed instruction can lead to better-managed outputs. This is the essence of prompt engineering  where you optimize prompts for specific tasks to get the most relevant and accurate results. All the advanced techniques are pretty much overkill except the few we will mention in this iteration.   High Learning Rate U+007C Louis-Franois Bouchard U+007C SubstackReal-world solutions for real-world problems. Leverage AI's potential with insider tips from specialists in the field highlearningrate.substack.com   2. Basics of Prompt FormattingPrompt formatting is extremely important for the model to understand what you want. Use bullet points or clear numbered lists to give your instructions or steps to follow. Just think of drafting the best Slack message for a new task for your colleague  and you should be good.   Most importantly: reread yourself and use Grammarly! Dumb typos transform words and affect how the models interpret them.   There are tools like Anthropics Generate a Prompt feature or Coheres Prompt Tuner that can help. These tools optimize your initial prompt  making it more detailed and clear to improve performance on whatever task you want. But you still dont want to depend on these tools and learn the essentials of prompting first.   For instance  a simple prompt like Complete sentences users will give you might be completely transformed into the following using Anthropics Generate a Prompt feature:    You are tasked with completing sentences that users will provide to you. Your goal is to finish the sentence in a way that is grammatically correct  coherent  and contextually appropriate.Here is the incomplete sentence: <incomplete_sentence> {{INCOMPLETE_SENTENCE}} </incomplete_sentence>To complete the sentence: 1. Carefully read the incomplete sentence and understand its context and tone. 2. Consider multiple possible endings that would make sense given the beginning of the sentence. 3. Choose the most appropriate and natural-sounding completion. 4. Ensure that your completion maintains the grammatical structure and tense of the original partial sentence. 5. Make sure the completed sentence is coherent and conveys a clear message.Provide your completed sentence within <completed_sentence> tags. The completed sentence should include both the original incomplete part and your addition  forming a full sentence.Now  please complete the given sentence:<completed_sentence> 3. Advanced Prompting TechniquesAs we said  there is no such thing as an advanced prompting technique. Just learn to chat with LLMs and get what you want with trial and error.   The best thing you can do is:   Be clear.Be concise.Ask the model to give its reasoning steps.Iterate (chain) with the model.Heres a bit more detail (with the proper names) about those only techniques you need to know   Zero-shot Prompting  aka Do thisThis is simply clearly telling what you want. Instruct the model without providing any examples. Its useful for straightforward tasks where the model has sufficient pre-trained knowledge:   Example: Classify the following text as positive or negative: I had a great day!Output: Positive.Note: Zero-shot comes from literature and it is used to describe what a model is capable of without any additional information. Its a way for scientists to describe the raw capabilities of a model. A fancy word for simple concept.   Few-shot Prompting  aka Here are some examplesFew-shot prompting is the best thing you can afford to do without retraining a model. It enhances the models ability to perform a task by providing a few examples (e.g. question-answer pairs) alongside the main prompt. This specificity helps the model understand the task better:   Format:Q: <Question>? A: <Answer>Q: <Another Question>? A: <Another Answer>We usually give 35 examples of the question and/or answer  and it tells the model how it should behave. This approach is the best bang for your buck to execute a new task the model wasnt trained to perform.   Chain-of-Thought Prompting  aka Think before actingChain-of-thought (CoT) prompting is probably the best method to make your LLM more intelligent. It does wonders. In CoT  we prompt the model to break down its reasoning steps. Clearly  the model is prompted to solve problems step-by-step  which is particularly useful for complex tasks like mathematical reasoning or generating comprehensive text summaries:   Prompt: Lets think through this step by step to solve the math problem: What is 23 + 56?Output: First  we add 20 and 50 to get 70. Then  adding the remaining 3 and 6 gives 9. So  70 + 9 equals 79.It basically acts as a manual mechanism to replicate our thinking process  just like we would think before saying our final answer. The model generates the text bit by bit  and each time it generates a new word (aka token)  it is added into the context along with the original prompt. This dynamically updated context helps the model think by decomposing the task step by step. This ultimately means that when you prompt a model  you force it to generate additional knowledge before answering and using it.   So when you ask the model to Think before acting  all the generated intermediate text (which are usually the initial steps or plan of action) are in its context  helping it understand the request better and plan before giving its final answer. Something all humans (should) do!   Chain Prompting  aka ChattingChain prompting just means iterating with the model. It is basically going back and forth with the AI to improve or correct its answer. You can do this either manually or with automated prompts. This has a similar goal as CoT but in a more dynamic way. The model will have more and better context  again allowing it to reflect back. It usually either uses yourself  other LLMs or APIs to discuss and get new outputs. It also allows you to add more dynamic content in the prompts depending on how the discussion (or exchange) advances.   Retrieval Augmented Generation  aka Search before answeringYou can draw a parallel with Retrieval-Augmented Generation (RAG). RAG is just about retrieving the most relevant information in a database before prompting an LLM. Then  you simply add this retrieved information along with the user question in the prompt. Here  we basically add useful context to the initial prompt before sending it to the model.   Prompt: Here is some context to answer the user question: <Retreived information>. Answer this question: <question>This helps the model answer the users question with specific knowledge. You can add as much relevant text as the model can handle.   Obviously  some functions allow you to use the Internet  which is the same as an RAG database  but it is the Internet.   For example  with ChatGPT  you can ask the model to use its web search tool before answering. This is really efficient if the response you seek needs up-to-date information.   Prompt: Which country has the most Olympic gold medals so far? Use the web search tool before answering.4. Output OptimizationBesides the prompt  there are other methods to improve output content quality and output structure.   For better content  you can adjust the temperature parameter to control randomness: lower values for more deterministic outputs and higher for more creative ones. You can also implement self-consistency (aka choose the most frequent answer) by prompting the model multiple times with the same input and selecting the most chosen response.   Regex checks after the generation can be used to ensure the model output respects a certain format. For example  you could hide the generation of a URL for security reasons if you build an application for your customers by spotting the http(s)://www or identifying a domain like towardsai.net. Another example would be to check if the output respects the JSON format.   Constrained sampling (aka blacklist words) is another similar concept that can be used where you tell the model which word or part of words to blacklist from the vocabulary of an LLM at generation time. With this method  the model wont be able to produce the blacklisted words and  therefore  can only generate desired words. The approach allows precise control over the output format with minimal performance impact because it simply filters words during generation (compared to post-generation  which could be done with the regex check).   Note: This method requires total access to the model. You can use llama.cpp to apply this technique with an open-weight model like Llama 3  but it cannot be used with an API-accessed model like GPT-4o.   With OpenAI and most other big LLMs  you can use tool (function) calling. Not all models can do that since training the model in a specific way is required. In JSON mode  LLMs are trained to generate outputs formatted as valid JSON  while function calling allows you to provide a function signature  and the model then returns the arguments to call that function in valid JSON format.   When experimenting with these approaches  consider not only the trade-offs between creativity  accuracy  and structure but also the capabilities of your chosen LLM. For instance  you might combine temperature adjustment and self-consistency to improve content  then apply an appropriate structuring method based on your LLMs capabilities and your specific needs  which will change if you switch from Llama to Claude.


Uncovering K-means Clustering for Spatial Analysis

Def- Underrated-adjective rated or valued too low- Merriam Webster.   Underrated  unappreciated or underhyped are terms that get thrown around to suggest something that does not get the recognition it deserves. Sometimes it is used to describe someone who does not get the public attention he deserves despite being very effective in their profession  this could be a persons biased opinion.   For example  I think that NBA basketballer Leonard Kawhi is the most underrated and criminally underhyped player of all time. Rapper Nathan John Feuerstein  also known as NF is highly underrated as both do not fit the perception of modern-day images of athletes and rappers.   The same could be said about some machine learning algorithms which are not talked about with excitement as they should be  as we are reaching the golden age of Artificial Intelligence and machine learning where some algorithms will be propped up while others may fall by the wayside of irrelevance due to this fact.   One such algorithm is K means which is known as an unsupervised algorithm and has become widely used but has not reached the popularity of random forest and K nearest- as I continue writing and researching on machine learning algorithms and their impact on the spatial sector- let us have a look at k means and what it offers to GIS pros.   What is K Means Clustering   K-Means is an unsupervised machine learning approach that divides the unlabeled dataset into various clusters. The purpose of this article is to examine the principles and operation of k-mean clustering as well as its application especially when it comes to geospatial analysis and its implication   Unsupervised machine learning algorithm as it is commonly referred to is the process of teaching an algorithm to work on unlabeled  unclassified data without human intervention. In this scenario  the machines task is to arrange unsorted data based on parallels  patterns  and variances without any prior data training.   K stands for clustering  which divides data points into K clusters based on how far apart they are from each others centres. The cluster centroid in the space is first randomly assigned.   To process the learning data  the K-means algorithm in data mining starts with a first group of randomly selected centroids  which are used as the beginning points for every cluster  and then performs iterative (repetitive) calculations to optimize the positions of the centroids.   How it Works   A clusters centroid is a set of characteristic values that together define the groups that are formed. The type of group that each cluster represents can be qualitatively interpreted by looking at the centroid feature weights.   Data assignment: The centroid  or centre collection of features  creates and defines each cluster. The closest centroid for each data point is then determined using a distance function of choice.   Update of the centroids: Following the assignment of all data points  the centroids are recalculated by averaging all the data points assigned to that cluster.   Repetition: Until a certain stopping condition is satisfied  such as no changes are made to clusters  the total distance is minimized  or a maximum iteration threshold is achieved  this assignment and update process is repeated.   K means for Spatial Analysis   Geographical data can be divided into k distinct clusters using the iterative K-means clustering algorithm. This is done by repeatedly assigning each data point to the closest centroid  recalculating the centroids as the mean of the assigned points  and repeating these steps until the centroids stabilize. This allows for the identification and interpretation of spatial patterns  such as market segments  urban land use types  environmental zones  and public health hotspots  while taking into account variables like distance metrics  data scaling  and geographic constraints to guarantee insightful and useful information.   Because of its scalability  it can manage enormous volumes of spatial data and is therefore appropriate for a variety of applications at both local and global sizes. GIS experts can find hidden insights in spatial data by utilizing K-means advantages  which will ultimately result in superior decision-making and outcomes for a variety of spatial analytic tasks.   It can be used for: -   Development and Urban Planning-Land Use Analysis: K-means assists city planners with resource allocation and zoning restrictions by classifying metropolitan areas according to land use types (residential  commercial  and industrial).   -Smart City Initiatives: K-means facilitates the development of smart city projects by improving infrastructure and services by grouping sensor data (from sensors measuring pollution or traffic  as example).   2. Disaster Management   Risk assessment: By identifying high-risk locations through K-means clustering of historical disaster data  disaster preparedness and mitigation planning are aided.   Resource Allocation: When responding to a disaster  grouping the impacted areas helps to prioritize the distribution of resources and rescue efforts.   3. Public health   illness Outbreak Detection: Public health professionals can identify regions with high illness incidence by clustering health data. This allows for focused treatments and effective resource distribution.   Healthcare Accessibility: By identifying underserved areas and examining the spatial distribution of healthcare services  K-means helps guide policy for improved healthcare access.   4. Real Estate   Property Valuation: Accurate property valuation and market analysis are aided by clustering property data according to features such as location  size  and amenities.   Development Planning: By using spatial clustering  real estate developers can pinpoint new trends and possible hotspots for development.   5. Transportation and Logistics   Route Optimization: By helping to cluster delivery points  K-means facilitates more effective routing and lowers transportation expenses.   Traffic Management: Cities can enhance traffic flow and better control congestion by clustering traffic data.   Snippet   Open your Google Earth engine   / import the satellite data from the European Space Agency var S2 = ee.ImageCollection(COPERNICUS/S2); //filter for Dubai S2 = S2.filterBounds(Dubai); print(S2); //filter for date S2 = S2.filterDate(2020-01-01  2020-05-11); print(S2); var image = ee.Image(S2.first()); print(image) var image = ee.Image(S2.first()); print(image) //Map.addLayer(image {min:0 max:3000 bands:B4 B3 B2}  Dubai); Map.addLayer(image {min:0 max:3000 bands:B8 B4 B3}  Dubai); // Create training dataset. var training = image.sample({ region: Dubai  scale: 20  numPixels: 5000 }); // Start unsupervised clusterering algorithm and train it. var kmeans = ee.Clusterer.wekaKMeans(5).train(training); // Cluster the input using the trained clusterer. var result = image.cluster(kmeans); // Display the clusters with random colors. Map.addLayer(result.randomVisualizer()  {}  'Unsupervised K-means Classification'); // Export the image to Drive Export.image.toDrive({ image: result  description: 'kmeans_Dubai'  scale: 20  region: Dubai });If you are enjoying this article please consider supporting my work and fuel my creativity by buying me a coffee  as Im not eligible for the Medium Partner Program but your contribution makes all the difference  any amount will do  Thanks.   Conclusion   K-means clustering has a significant impact on spatial analysis by providing a flexible and effective tool for finding patterns  maximizing resources  and making defensible decisions in a variety of contexts  including business strategy  public health  and environmental monitoring in addition to urban planning. It is a priceless tool in todays data-driven decision-making processes due to its efficiency in managing huge spatial datasets and delivering insightful analysis.


Google Does it Again

Google Deepmind has done it again. And this time  its a double win.   They have presented AlphaProof and AlphaGeometry 2  models that have achieved silver medalist-level performance by solving challenging International Mathematical Olympiad problems  competing with the best humanity has to offer.   This is a highly interesting piece of research  as it gives insight into the cutting-edge of what the AI industry represents in terms of mathematical and reasoning progress.   At the same time  it serves as a clear indication of what Google is working on in the future: cracking the code to create what would certainly take us close to a real super AI: a depth generalizer.   But what do I mean by that?   Learning about AI is useless unless it helps you make better decisions. This is what my newsletter intends to achieve  a newsletter written for AI analysts  strategists  investors  and Leaders that looks to answer the most pressing questions in AI:   Are we in a bubble?; Is AI actually Intelligent?; or why does Taiwan matter so much?   In a nutshell  having the overall technological  geopolitical  and economic view of the industry in weekly  easy-to-follow reports.   U+1F3DDU+1F3DD Subscribe today below:   TheTechOasisThe newsletter to stay ahead of the curve in AIthetechoasis.beehiiv.com   Depth vs BreadthAt some time in your AI journey  you may have wondered: what made ChatGPT different from everything that came before?   The field of AI is older than most of us and can be traced back to the Dartmouth Summer Research Project on Artificial Intelligence in 1956.   That said  Alan Turing first introduced the notion of AI in its historically significant Computing Machinery & Intelligence in 1950. One way or the other  its safe to say that AI has become prominent in our lives over the last two years.   A man way ahead of his time.   To put into perspective how advanced to his time Alan Turing was  he conceived machine thinking as The Imitation Game. Well  74 years later  most AI systems  including ChatGPT  are literally the embodiment of this idea.   And only a handful of AI systems  casually the ones we are looking at today  are not based on pure human imitation  making our story today even more relevant.   Before the advent of Large Language Models (LLMs) like ChatGPT  all AI was deeply narrow. In other words  we trained models to perform one task as well as possible  but these models could not work well in more than one task.   This idea  called depth  governed the industry for decades  not because that was everything we wanted  but because generalization  when a model can perform various tasks  was just a pipe dream.   LLMs like ChatGPT sort of solved that problem. However  this has come at a sacrifice.   The ChatGPT FallacyWhile this weak generalization we have achieved with ChatGPT (they still completely fail in tasks where their memorization capabilities cant help them) has been extremely economically fruitful to those building these models (especially Big Tech  which has added around $7 trillion combined since November 2022 excluding Tesla)  its also a setback in other regards.   For reference  $7 trillion is more than the combined value of the stock markets of the UK  Germany  and Spain  and you would still have a spare trillion dollars.   Despite what markets will indicate based on valuations  LLMs are good at many things but great at none.   In other words  we have sacrificed task-specific greatness in lieu of models that  while they can write a Shakespearean poem and talk to you about nuclear fission  both responses will be surprisingly average.   This is what I call The ChatGPT Fallacy  or the greatest mistake one can make when using ChatGPT.   When testing the model's prowess  most people will often ask it what they dont know themselves instead of asking it something they have the criteria to know if it's good. When tested in the latter form  the real limitations of ChatGPT become apparent quickly.   In more technical terms  our best models are currently losing depth (per-task prowess) for better generalization (doing many tasks but being meh at them).   While this turn is understandable from a business perspective (markets pay handsomely)  it has had tragic consequences  and we might be going backward.   From AlphaGo to ChatGPT and BackBefore ChatGPT came into our lives  the most impressive AI the world had ever seen was the AlphaGo model family  deep neural networks (just like ChatGPT in that sense) that achieved superhuman capabilities in the game of Go  a Chinese board game (with AlphaZero being the state-of-the-art not only in Go but also in Shogi and Chess).   It even defeated Lee Sedol  the champion at the time  in a historical event that even led to documentaries on the feat.   But how do Alpha (Go and Zero) work?   They are based on Monte Carlo Tree Search  but on steroids. The idea is that for each new movement  the models explore thousands or even millions of possible outcomes for each movement they could make  estimating the expected maximum cumulative reward to decide on the one that its probabilities suggest is the best.   In a way  you can think of these models as machines that look ahead of their current move to choose the best one. Interestingly  this is the precise capability researchers are trying to instill in LLMs to improve their reasoning  using methods similar to MCTS  like Tree-of-Thought  depicted below  where the LLM explores different solution paths before deciding on one.   For more detail on the convergence of both worlds on the path to conquering true intelligence  read my deep dive Is AI Really Intelligent?   Although the AlphaX models could only play Go (and a handful of other board games)  they were better than any human in history at them. But as money has flown into LLMs  the quest for superhuman AIs like these has mostly stalled in recent years.   Obviously  merging both worlds is the ultimate goal  and here is where our fellows at Google come in: Can we create an LLM with star capabilities at many tasks?   On the Measure of RewardsDespite what many may believe  thanks to OpenAIs fame  Google Deepmind is head and shoulders above the rest when it comes to achieving depth.   This is because they are great at Reinforcement Learning  a field where robots are incentivized to perform actions in an environment to achieve a goal by punishing or rewarding them depending on the outcome of each action.   As we need to measure the rewards continuously  this method requires an auxiliary model  a reward model  to perform this evaluation. In a way  RL is like playing a game  but a game in which the model learns based on this reward feedback.   As you may have guessed  the quality of the outcome depends heavily on choosing the correct reward and punishment mechanisms  which is a really hard thing to define  especially in robotics.   In particular  NVIDIA has been proposing AIs that build reward functions for some time  achieving impressive results we covered in my newsletter a while back.   In a particular set of cases  like AlphaGo  this method can create superhuman AIs that  by performing self-improvement  sometimes known as self-play  or using its outputs as feedback  can transcend human limitations and become much better than us at certain tasks.   A good example is this video we shared in this newsletter months ago  a robot that  in just six hours of training  becomes superhuman at the Labyrinth game.   Well  now Deepmind is experimenting with this idea in mathematics theorem proving  and the results are humbling for humans.   In The Conquest of Maths ReasoningGoogle Deepmind has presented two models:   AlphaProof  a model that excels at proving mathematical statements using Lean  a programming language that aims to work as a proof assistant.AlphaGeometry 2.0  a new version of a model I covered in detail in my Medium blog that excels at geometry theorem proving.And  in both cases  LLMs play a crucial role.   AlphaProofTo design AlphaProof  they created a self-improvement loop of theorem proving with AlphaZero  the model we discussed earlier  by using an LLM to draft the mathematical statements in a formal way that AlphaZero can then try to prove.   Then  for those theorems that AlphaZero successfully proves  they use them to reinforce that behavior  aka they use them as a signal to improve the model.   The reason for this is that adequate data for such type of training is almost non-existent.   Thus  using Geminis capabilities to rewrite data (depicted as the Formalizer network above)  they created 100 million formal problems on which they trained AlphaZero.   So  in a way  AlphaProof does the same thing that AlphaZero does when playing Go  but instead of exploring the next best movement  its capable of exploring the next best proof step in its path to finding the proof to a given theorem.   AlphaGeometry 2As for the latter  AlphaGeometry is an even more interesting model. In a nutshell  its a neurosymbolic AI model  a combination of an LLM (Gemini) and symbolic engines.   But what do I mean by that?   A neurosymbolic system is an AI system that combines a neural network  in this case  an LLM  with hard-coded systems of human-written code that can perform accurate calculations or actions if we constrain the problem enough.   For instance  a symbolic engine might be a mathematics software written by humans that takes in a set of constraints and calculates the output (like being provided the length of two sides of a right triangle and using the Pythagoras theorem to compute the third side).   And what is the role of the LLM here? They search.   But search what?   Symbolic engines are lists of conditions written by humans in the form of if x happens  do y. They are the epitome of what we call Symbolic AI  which  in reality  is just machines imitating intelligent human actions in highly constrained environments.   But heres the thing: when facing an open geometry theorem problem  there are potentially infinite ways to approach the solution.   Symbolic engines cant search; they are limited by the number of scenarios that the humans who coded that engine thought of. In other words  when faced with open problems  they dont work.   So what does Gemini (Googles LLM) do?   When faced with an open problem like proving a geometry theorem  it suggests what researchers call auxiliary constructions (depicted in red and blue below  with the black lines being the original problem)  cues added to the problem that constrain the space of possible solutions.   For instance  in the image below  Gemini proposes computing point E (the right angle in triangle AEB)  which is then used to compute other triangles that narrow the problem and facilitate the solution.   In laymans terms  AlphaGeometry 2 works as follows:   Gemini suggests possible solution paths to the symbolic engines performing the computation. Simply put  we are narrowing down an open problemThen  the symbolic engines can compute and test whether its sufficient to prove the theorem.In a nutshell  this method breaks intelligence into two parts: idea generation  and idea verification  which is a very common way of defining intelligence in the AI industry these days. First  a system proposes possible ways of solving a problem  and the second system computes and verifies whether that path is correct.   However  Google Deepmind takes a slightly different approach from most (at least in this research).   While most AI enthusiasts think of solving intelligence as an end-to-end neural network (think OpenAI)  where idea generation and verification are both done by neural networks (mainly LLMs)  here Google suggests that the verification part is done by human-written code  which is what neurosymbolic AI is.   While we cant know which of the two methods will prevail  the Olympic silver medal tells you all you need to know about how powerful this method is.   And  once again  Google is openly telling the world: When it comes to deep AI  everyone else is in our rearview mirror.   And now what?In a way  these two models exemplify what  in a nutshell  is the next frontier of AI: combining LLM-powered search with RL-trained models that excel in depth (aka  excel at specific tasks).   Unlocking this paradigm at scale could create the first deep generalizer that  akin to humans  can not only perform several tasks but  upon facing a complex problem  can search the space of possible solutions until it finds the best one.   To me  this sounds a lot like what most people think of AGI when trying to define it.   Markets-wise  its unclear how Google will monetize this; these models seem more like research projects. However  the very smart use of neurosymbolic systems  which not only learn faster but are cheaper to run  suggests that Google could release AlphaGeometry 2 to academia to enhance the works of mathematicians worldwide.   That said  I feel research like this should have a major effect on markets  but it doesnt  as investors usually look at numbers and what a handful of biased tech tycoons say in deeply orchestrated interviews.   However  when considering these investors sole job is to predict the future and make a return  seeing a company like Google present research like this should be extremely optimistic and even a major signal that Google might soon release a commercially available AI mathematician copilot.   For business inquiries on AI strategy or analysis  reach out at nacho@thewhitebox.ai   If you have enjoyed this article  I share similar thoughts in a more comprehensive and simplified manner for free on my LinkedIn.   If preferable  you can connect with me through X.


Building and Extending Your Decision Tree: A Hands-On Guide

Introduction   This post explores decision trees and guides how to build one from scratch. Ill start with the basics and gradually move on to more advanced techniques to enhance the decision tree.   Ill begin with a simple example to explain the basics of decision trees. Then  Ill delve into the mathematics behind them  covering key concepts like entropy and Gini impurity. Ill also introduce the soft trees using the logistic function.   After covering the theory  Ill dive into coding to show you how to build your decision tree  without using pre-built libraries. Finally  Ill explore advanced techniques to optimize the trees performance  such as using KS statistics and combining different metrics.   By the end of this guide  youll have a solid understanding of decision trees and the confidence to build and tweak your AI models. Lets get started!   Understanding the Basics with a Simple ExampleLets dive into the basics of decision trees using a simple example. Imagine we have data from 1000 individuals with different ages (our input variable x)  and we want to predict whether they are employed ( target variable Y  binary: 1 for employed  0 for not employed). The goal is to build a model f(x)=Y that predicts employment status.   To start  we need to find the best cut-off age that divides our data into two groups: those above the cut-off age and those below it. This split should maximize the difference in employment rates between the two groups.   For instance  lets assume that age 30 is the best cut-off point. This means the employment ratio for people older than 30 and those younger than 30 shows the largest difference. We can then create a simple decision rule: if a persons age is greater than 30  they are more likely to be employed  and if they are 30 or younger  they are less likely employed.   Mathematically  we can represent this decision rule as follows:   This is a step function where f(x) predicts 1 (employed) if the age x is greater than 30 and 0 (unemployed) if x is 30 or less. This simple model illustrates how a decision tree works by splitting the data at the optimal cut-off point to make predictions.   In reality  decision trees can handle multiple variables or a vector:   The tree considers different split points to find the best way to divide the data. The key aspects of building a decision tree are 1) choosing the best variable to split on  and 2) finding the optimal split point. Ill discuss these aspects in the next paragraph.   Mathematical Concepts and Extensions for Decision TreesUnderstanding Decision TreesLets dive into the math behind decision trees and investigate how they can be extended. Imagine a decision tree as a series of steps that divide the data into different regions  each associated with a different outcome. For instance  in our earlier example  we used age (x) to predict employment status (Y). We started with a simple decision rule that looked like this:   When extending this to multiple variables  the decision tree model can be expressed as:   Each decision node in the tree represents a condition or a set of conditions based on the features x_1  x_2    x_n. The model is built by recursively splitting the data according to these conditions. For example  a step function for two variables  age (x_1) and income (x_2)  might look like this:   For instance  if we have two variables  age (x_1) and income (x_2)  each with its respective cut point  the decision function would like this:   We would use the cut tree method to solve Y = F(x_1  x_2 ). Think of it like slicing a cake. We need to make the best cuts to ensure each piece has the right balance of ingredients.   Here is how it works: First  we identify the most significant variable for splitting. This decision is similar to choosing where to make the initial cut on a cake. Should the division be based on age  income  or another factor?   Impurity MeasuresThis involves selecting the best variable and the optimal split point based on criteria such as entropy  Gini impurity  and information gain.   Entropy is a measure of impurity or randomness in the data  calculated as:   Here S is the dataset  c is the number of classes  and p_i is the proportion of instances in class i. Imagine we split our data and make each group as pure as possible. Entropy measures the randomness or impurity. Lower entropy means a purer group.   Gini impurity is another measure of impurity given by:   This is another way to measure purity  calculating how often youd randomly pick the wrong item if you randomly picked from a group. Lower Gini impurity means fewer mistakes.   Information gain measures the reduction in entropy or impurity after a split:   This tells us how much weve improved the purity of the data after a split. It measures the reduction in entropy. A higher information gain means weve made a good split.   These metrics are important because they help the tree figure out the best places to make splits  ensuring that each split makes the groups as similar as possible. For example  if you predict whether people will buy a product based on age and income  you want to create groups where people are more likely to make similar decisions.   You might question how to use these impurity measures in practice. Dont worry; Ive got you covered. Heres a summary based on my experience and research:   When to Use Each Criterion?   Entropy: Think of entropy as the meticulous organizer. Use it when you want to be precise about how mixed your groups are. Its perfect for tasks where you care about the exact distribution of your classes.   Gini Impurity: Gini is your go-to for quick  reliable results. It gives you a good measure of purity without much fuss. when solving binary classification problems  its less heavy on the calculations and gets the job done efficiently.   Information Gain: Use this when using algorithms such as ID3  C4.5  or C5.0  since this measure is about understanding how much each split helps you. Its also useful for observing the added value of each decision in the decision tree.   Smooth Function for Soft TreesA natural question is: is there any smooth the-step function for the cut tree method?   This is important for neural networks since traditional decision trees are non-differentiable step functions  and struggle in training neural networks models. Using smooth soft tree-like functions in NN as activation functions will enable backpropagation and gradient descent. In addition  the smooth soft tree can facilitate training and improve performance by providing probabilistic outputs. The soft prevents overfitting by smoothing decision boundaries  creating a more stable and generalizable model.   To introduce the concept of a soft tree  we can use a logistic function to smooth the step function. The logistic function is defined as:   Where s is the cut point  B is a large positive value (penalty) defining the steepness. This function approximates a step function when B is large  as the above equation quickly approaches 0 for x<s and 1 for x>s. This allows us to assign probabilities to the tree branches  effectively transitioning from a soft to a hard decision tree.   Incorporating this into our tree  the prediction function can be updated as follows:   if soft is chosen  the prediction is based on the weighted average of p and the hard trees assignment   This approach introduces the concept of a soft tree  where the decision boundaries are smooth rather than sharp.   Relationship Between Impurity MeasuresLets examine how the soft trees function relates to the hard trees impurity measures. In logistic regression  the probability of the positive class is modeled using the logistic function:   Given this model  the likelihood function for a single data point (x  y) is:   The overall likelihood for the dataset is:   Taking the logarithm  we obtain the log-likelihood:   Substituting p from the logistic function:   This shows that maximizing the likelihood function in logistic regression aims to find the best coefficients A and B that separate the classes effectively. This is similar to finding the split that minimizes impurity in Gini or entropy terms.   We first compare the Logistic Regression Likelihood and Gini Impurity:   If the data is well-separated  the logistic regression models decision boundary will minimize overlap  resulting in pure subsets (low Gini impurity). As p approaches 0 or 1 (pure classes)  Gini impurity 2p(1p) approaches 0  indicating pure nodes.   We then compare the Logistic Regression Likelihood and Entropy:   The log-likelihood function has terms p log(p)  (1  p) log(1  p)  similar to the entropy formula. This means that by maximizing log-likelihood  we are also reducing entropy. We aim for confident classifications  where p is close to 0 or 1. When p is near 0 or 1  the entropy p log(p)  (1  p) log(1  p) becomes 0  indicating pure nodes.   Understanding and implementing soft tree methods with the logistic function is also important in deep learning and KolmogorovArnold Networks (KAN)  as it bridges the gap between decision trees and neural network architectures  enabling more flexible and powerful models.   In the next section  Ill discuss the code implementation for traditional decision trees. This can provide you with a clear understanding of their mechanics. In addition  it will help readers develop their trees and understand how the model works. Ill also show why a parametric approach with the logistic function is important.   Implementing a Traditional Decision Tree from ScratchI will study the code for building a decision tree classifier from scratch  without relying on pre-built libraries like scikit-learn. This hands-on approach will help you understand the underlying mechanisms of decision trees and how to implement them yourself. Lets dive into the code block by block  focusing on the key parts and more complex aspects of the tree growth and best split calculations.   Gini Impurity Calculation   def gini_impurity(y): m = len(y) return 1.0 - sum((np.sum(y == c) / m) ** 2 for c in np.unique(y))The function calculates the Gini impurity for a given set of labels y. Here  we use this measure in split-point searching.   Finding the Best Split   def best_split(X  y): best_gini = 1.0 best_idx  best_thr = None  None m  n = X.shape for idx in range(n): thresholds = np.unique(X[:  idx]) for thr in thresholds: left_mask = X[:  idx] < thr right_mask = ~left_mask if sum(left_mask) == 0 or sum(right_mask) == 0: continue gini = (sum(left_mask) * gini_impurity(y[left_mask]) + sum(right_mask) * gini_impurity(y[right_mask])) / m if gini < best_gini: best_gini = gini best_idx = idx best_thr = thr return best_idx  best_thrThe function finds the best feature and threshold to split the data by minimizing the Gini impurity. Here's a detailed breakdown of its process:   Initialization: best_gini is initialized to 1.0  representing the worst possible impurity. best_idx and best_thr will store the best feature index and threshold for splitting.Iterating Over Features and Thresholds: The function loops over each feature and each unique threshold value for that feature. For each combination:Masking: It creates masks to separate the data into left and right subsets based on whether the feature values are below or above the threshold.Gini Calculation: The Gini impurity for the split is calculated by weighting the Gini impurities of the left and right subsets by their sizes.Updating Best Split   If the current splits Gini impurity is lower than the best observed so far  the function updates best_gini  best_idx  and best_thr.   class Node: def __init__(self  gini  num_samples  num_samples_per_class  predicted_class): self.gini = gini self.num_samples = num_samples self.num_samples_per_class = num_samples_per_class self.predicted_class = predicted_class self.feature_index = 0 self.threshold = 0 self.left = None self.right = NoneThe class represents a node in the decision tree. It stores:   Gini: The Gini impurity of the node.num_samples: The number of samples at the node.num_samples_per_class: The distribution of samples per class.predicted_class: The class prediction at the node.feature_index and threshold: Used to store the best feature and threshold for splitting.left and right: Pointers to the left and right child nodes.Decision Tree Classifier   class DecisionTreeClassifier(BaseEstimator  ClassifierMixin): def __init__(self  max_depth=None): self.max_depth = max_depth def fit(self  X  y): # Check for NaN or infinite values in the data if np.any(np.isnan(X)) or np.any(np.isnan(y)): raise ValueError(Input data contains NaN values.) if np.any(np.isinf(X)) or np.any(np.isinf(y)): raise ValueError(Input data contains infinite values.) self.classes_ = np.unique(y) self.n_classes_ = len(self.classes_) self.n_features_ = X.shape[1] self.tree_ = self._grow_tree(X  y) return selfThe class is the main class for our decision tree. It inherits from BaseEstimator and ClassifierMixin to ensure compatibility with scikit-learn. Heres what each method does:   Initialization: Sets the maximum depth of the tree.Fitting the Model: Checks for NaN or infinite values in the input data  initializes class and feature information  and starts the tree-growing process.Growing the Tree   def _grow_tree(self  X  y  depth=0): num_samples_per_class = [np.sum(y == i) for i in range(self.n_classes_)] predicted_class = np.argmax(num_samples_per_class) node = Node( gini=gini_impurity(y)  num_samples=X.shape[0]  num_samples_per_class=num_samples_per_class  predicted_class=predicted_class  ) if depth < self.max_depth: idx  thr = best_split(X  y) if idx is not None: indices_left = X[:  idx] < thr X_left  y_left = X[indices_left]  y[indices_left] X_right  y_right = X[~indices_left]  y[~indices_left] node.feature_index = idx node.threshold = thr node.left = self._grow_tree(X_left  y_left  depth + 1) node.right = self._grow_tree(X_right  y_right  depth + 1) return nodeThe method recursively builds the decision tree:   Node Creation: Initializes a new node with the Gini impurity  sample counts  and predicted class.Splitting the Node: It finds the best split if the current depth is less than max_depth. If a valid split is found:Left and Right Subsets: Creates subsets of the data based on the split.Recursive Calls: Recursively grows the left and right subtrees.Making Predictions   def predict(self  X): return np.array([self._predict(inputs) for inputs in X]) def _predict(self  inputs): node = self.tree_ while node.left: if inputs[node.feature_index] < node.threshold: node = node.left else: node = node.right return node.predicted_classThe predict method uses the helper function _predict to traverse the tree from the root to a leaf node  making predictions for each input:   Traversing the Tree: Starting from the root node  it moves to the left or right child based on the feature value and threshold.Leaf Node: Once a leaf node is reached  it returns the predicted class stored at the leaf.This custom implementation of a decision tree classifier provides a comprehensive understanding of how decision trees work. With this knowledge  you can tackle more advanced applications and customizations in machine-learning algorithms.   Building My Decision Tree with Enhanced Splitting CriteriaIn this section  I will demonstrate an extended decision tree method that incorporates the Kolmogorov-Smirnov (KS) statistic  entropy  and a combination of both to determine the best split. Additionally  we will introduce a soft decision tree factor  using the logistic function:   to assign probabilities to branches  making the decision tree more adaptable and robust.   Why Use KS?The KS statistic measures the maximum difference between the cumulative distributions of two samples  so  in this paper  I would use it to identify the most significant split points.   Traditionally  the Kolmogorov-Smirnov (KS) statistic is used to test the hypothesis that two samples come from the same distribution. It measures the maximum distance between the two samples empirical cumulative distribution functions (CDFs). Mathematically  the KS statistic for two samples F_1(x) and F_2(x) is defined as:   This method has also proved effective in distinguishing between two classes. So I make it a useful tool for decision trees.   Key Parts of the CodeHeres a closer look at the critical sections of the code:   Initialization: The class is initialized with parameters like depth  minimum samples per split  number of potential cut points  split criterion  and whether to use the 'soft' tree option.class DecisionTree(BaseEstimator  ClassifierMixin): def __init__(self  depth=10  min_samples_split=2  num_cut_points=100  split_way='entropy'  soft=False  B=5): self.depth = depth self.min_samples_split = min_samples_split self.num_cut_points = num_cut_points self.split_way = split_way self.soft = soft self.B = B self.tree = NoneBuilding the Tree: The method constructs the tree recursively. It splits the data into left and right subsets based on the best-split point determined by the _best_split method. A leaf node is created if the maximum depth is reached or the number of samples is below the minimum.   def _build_tree(self  X  y  depth): if depth == 0 or len(X) < self.min_samples_split: return self._leaf_value(y) feat_idx  threshold = self._best_split(X  y) if feat_idx is None: return self._leaf_value(y) left_idx = X[:  feat_idx] < threshold right_idx = ~left_idx left_tree = self._build_tree(X[left_idx]  y[left_idx]  depth-1) right_tree = self._build_tree(X[right_idx]  y[right_idx]  depth-1) return (feat_idx  threshold  left_tree  right_tree)Finding the Best Split: The _best_split method evaluates potential split points using KS  entropy  or both. It iterates over thresholds for each feature and calculates the KS statistic and entropy between the left and right splits. The split with the highest score is chosen.   def _best_split(self  X  y): best_score = -np.inf split_idx  split_thresh = None  None for feat_idx in range(X.shape[1]): thresholds = np.linspace(np.min(X[:  feat_idx])  np.max(X[:  feat_idx])  self.num_cut_points) for threshold in thresholds: left_idx = X[:  feat_idx] < threshold right_idx = ~left_idx if sum(left_idx) == 0 or sum(right_idx) == 0: continue if self.split_way == 'KS': ks_stat  _ = ks_2samp(y[left_idx]  y[right_idx]) score = ks_stat elif self.split_way == 'entropy': left_entropy = entropy(np.bincount(y[left_idx]  minlength=self.n_classes_) + 1e-10) right_entropy = entropy(np.bincount(y[right_idx]  minlength=self.n_classes_) + 1e-10) total_entropy = (sum(left_idx) * left_entropy + sum(right_idx) * right_entropy) / len(y) info_gain = entropy(np.bincount(y  minlength=self.n_classes_)) - total_entropy score = info_gain elif self.split_way == 'both': ks_stat  _ = ks_2samp(y[left_idx]  y[right_idx]) left_entropy = entropy(np.bincount(y[left_idx]  minlength=self.n_classes_) + 1e-10) right_entropy = entropy(np.bincount(y[right_idx]  minlength=self.n_classes_) + 1e-10) total_entropy = (sum(left_idx) * left_entropy + sum(right_idx) * right_entropy) / len(y) info_gain = entropy(np.bincount(y  minlength=self.n_classes_)) - total_entropy score = (ks_stat + info_gain) / 2 else: raise ValueError(fUnknown split_way: {self.split_way}) if score > best_score: best_score = score split_idx = feat_idx split_thresh = threshold return split_idx  split_threshPredicting with the Soft Tree: The _predict_proba method uses a logistic function to assign probabilities to each branch. This makes the tree 'soft'  allowing it to handle uncertainty better.   def _predict_proba(self  inputs  tree): if not isinstance(tree  tuple): return np.eye(self.n_classes_)[tree] feat_idx  threshold  left_tree  right_tree = tree prob = 1 / (1 + np.exp((inputs[feat_idx] - threshold) * self.B)) left_prob = self._predict_proba(inputs  left_tree) right_prob = self._predict_proba(inputs  right_tree) combined_prob = prob * left_prob + (1 - prob) * right_prob combined_prob /= combined_prob.sum() return combined_probHere are the results:   The results show that the KS statistic can boost decision tree performance. While the soft tree doesnt improve performance  it helps understand how to integrate probabilistic assignments into decision trees. This approach is useful for neural networks  where smooth activation functions are important for backpropagation and gradient descent. The soft tree method adds flexibility and potential for further optimization in machine learning models.   ConclusionI wrote this guide since decision trees are important tools in machine learning methods like XGBoost  LightGBM  and neural networks. While people often use pre-built packages like scikit-learn  understanding the underlying mechanics and math of decision trees allows for improvements and new algorithm creation.   By building a decision tree from scratch  you learn how splits are decided  how impurity measures guide these splits  and how the KS statistic can improve performance. I also introduce the concept of soft trees  which is useful for integrating probabilities and is important for neural networks.   Mastering decision tree basics helps you fine-tune and optimize models for specific tasks. It is also helpful for building advanced applications and customizations in machine learning projects.   The Python scripts are available in my GitHub repository: GitHub  datalev001/decsiontree.


Learn AI Security For FREE With These Amazing Resources

AI security is the career of the future.I have written about this MANY times and keep repeating it on auto-pilot to anyone who wants to future-proof their Cybersecurity career.   But where to start?   A common misconception amongst people is that you need to be super-technical or have a PhD in Data Science to learn AI security   The field is vast enough to accommodate people from both technical and non-technical backgrounds   There is no need to buy expensive courses  as the Internet has some amazing resources you can use.   Here are the ones I would recommend   1  NIST AI Risk Management FrameworkThe NIST Cybersecurity Framework has become an industry benchmark companies use to assess their security posture against best practices.   The NIST AI Risk Management Framework (RMF) is poised to do the same for AI Risks.   The NIST AI RMF is a tech-agnostic guidance developed to help companies design  develop  deploy  and use AI technologies responsibly.   NIST frameworks are well-trusted within the industry due to the rigorous validation they undergo from experts all across the globe   This framework is an excellent starting point for people  regardless of their technical background.   It provides a comprehensive approach to managing AI risks through key components such as governance  mapping  measuring  and managing AI systems.   More and more companies will use this framework to manage their AI risks as AI adoption ramps up.   If you find the framework too boring ( which it can be )  then I would recommend using the AI RMF playbook  which is an interactive companion piece to the framework.   It is much more streamlined and engaging than the framework  allowing you to filter those areas that are interested in   2  AWS GenAI Security Scoping MatrixI admit I am a bit biased  given that I currently work in AWS   But if you are interested in GenAI security  then the AWS GenAI Security Scoping Matrix is one of the best resources around   This three-part series helps you understand the different ways of assessing Generative AI risk and how they change depending on the model your company chooses   The concepts are not just restricted to AWS and can be applied to any provider   Highly recommended for those wanting to deep-dive into GenAI risks   3  OWASP Top 10 For LLMThe OWASP Top 10 is another industry benchmark for web application security.   So it was no surprise when they released their new top 10  this time focusing on Large Language Model Applications   As per OWASP   The OWASP Top 10 for Large Language Model Applications project aims to educate developers  designers  architects  managers  and organizations about the potential security risks when deploying and managing Large Language Models (LLMs).   Similar to their previous top 10s  this document lists the most critical vulnerabilities found in LLM applications.   It shows their impacts  how easy it is to exploit  and real-world examples   If you are a CISO or have security leadership responsibilities  it also comes with a great companion piece  the LLM Security & Governance Checklist.   The checklist helps you understand how to assess AI risks and implement an oversight program to mitigate them   4 MITRE ATLAS FrameworkThe previous frameworks I highlighted are great  but they can be too high-level for someone who likes to dive deep into the technicalities of AI attacks.   This is where ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) comes in.   As per their website   ATLAS is a globally accessible  living knowledge base of adversary tactics and techniques against Al-enabled systems based on real-world attack observations and realistic demonstrations from Al red teams and security groups.   As the diagram below shows  ATLAS demonstrates how attackers can compromise AI at each stage and what techniques are used.   An excellent resource if you want to become an AI pent-tester!   Their website also has a great primer on AI security  which you might want to review before you dive into the ATLAS matrix.   Thanks for reading this and good luck on your AI Security career !   If you are interested in acing your next Cybersecurity Interview then check out my Free Ebook HERE   Taimur Ijlal is a multi-award-winning  information security leader with over two decades of international experience in cyber-security and IT risk management in the fin-tech industry. Taimur can be connected on LinkedIn or on his YouTube channel  Cloud Security Guy on which he regularly posts about Cloud Security  Artificial Intelligence  and general cyber-security career advice.