Algorithms and AI: An Inside Look in GA

Since the launch of ChatGPT in November 2022 by OpenAI, along with other competing chatbots in this industry, such as Google’s BARD and Microsoft’s Bing Chat, a new race has emerged in developing Artificial Intelligence (AI) tools.

Whether it is used as a chatbot, for image recognition, design generation, or other applications, artificial intelligence aims to accomplish simple and complex tasks by emulating human logic. These tasks are completed through the execution of algorithms, which follow a set of operating instructions and mathematical calculations. As technology advances, AI continues to evolve, driving innovation and transforming various industries.

Artificial Intelligence AI

Artificial intelligence is a subdiscipline of computer science that seeks to enable computers to mimic human intelligence in order to solve specific problems. To achieve this, computer scientists create and develop new algorithms that increasingly simulate the cognitive abilities of humans. Currently, AI has specialized in various subfields, including:

Evolutionary Computation

This field encompasses a family of algorithms inspired by biological evolution. These algorithms are used to optimize complex problems with multiple variables and in continuous optimization systems (e.g., Genetic Algorithms).

Artificial Neural Network

It is inspired by biological neural networks where algorithms aim to learn from examples with known input and output data. These algorithms are characterized by their layers and the prior definition of weighting factors to handle a specific situation or problem. Due to these features, they are also considered a subset of Machine Learning (e.g., Deep Neural Networks).

Vision

It is a subfield that focuses on using vision techniques to identify objects by processing images or video frames. Scientists develop algorithms with the goal of enabling computers to understand the contents of an image.

Robotics

This subfield requires knowledge of Electrical, Mechanical, and Computer Science. Engineers seek to create autonomous robots capable of performing various tasks, ranging from simple object manipulation to specialized tasks (e.g., Intelligent Control, Autonomous Exploration).

Expert Systems

It focuses on designing and constructing algorithms that facilitate decision-making for project managers in dealing with complex problems requiring extensive knowledge and specific expertise (e.g., Decision Support Systems).

Speech Processing

Engineers focus on developing algorithms that have the capability to convert spoken language into text, understand human messages, and translate them into actionable responses (e.g., Speech Recognition and Production).

Natural Language Processing

This interdisciplinary subfield combines linguistics and computer science. Algorithms in this field aim to make human languages understandable, allowing programmed systems to interpret and comprehend texts and engage in clear dialogue with humans. (e.g., Machine Translation, Natural Language Generation, Grammatical Error Correction).

Machine Learning

It is a well-known subfield of AI that focuses on studying and developing methods to train machines to learn from data and make predictions (e.g., Decision Tree Learning and Version Space Learning).

Algorithms for Constructing AI

Once you have knowledge of algorithm development in the context of AI, it’s possible to delve deeper and explore their classification. AI algorithms can be categorized into various groups, with the main three being:

Supervised Learning

These algorithms are primarily characterized by their reliance on human intervention to label input data. They are commonly used for classification and numerical regression, which are essential techniques in data analysis and decision-making processes. Examples of such algorithms include Decision Trees, Random Forest, Support Vector Machines, Linear Regression, polynomial regression, and Logistic Regression.

Machine Learning Algorithms for Classification. Source.

Unsupervised Learning

Unlike the previous type, this category of algorithms does not rely on data labeling to build models and discover relationships. These algorithms must be designed to identify patterns and make predictions without explicit instructions. Examples of such algorithms include K-means Clustering, Hierarchical Clustering, and Gaussian Mixture Model.

Note that algorithms like Neural Networks can be utilized as supervised and unsupervised learning paradigms.

Deep Neural Network. Source

Reinforcement Learning

This type of algorithm is characterized by learning from feedback obtained through iterative operations of action and result. In other words, these algorithms evaluate multiple possibilities for a problem through trial and error, receiving a positive reward each time they approach the best solution. Their objective is to maximize the reward for a given situation. Some frameworks include Markov Decision Process (MVP), Bellman Equations, Dynamic Programming, and Q-learning.

Optimal Path Routing. Source.

As you can see, each of these categories encompasses a wide variety of algorithm types, making it impractical to cover all of them in a single post. Therefore, we have decided to start a series of blogs where we will provide detailed information about each algorithm. Here, we explore an algorithm employed in problem optimization and machine learning model building.

Evolutionary Computation: Genetic Algorithms

Genetic Algorithms (GA) belong to the subset of artificial intelligence known as evolutionary computing. These algorithms are designed based on natural selection and genetics principles, and their objective is to minimize or maximize an objective function. In simpler terms, the variables to be optimized are represented by strings of binary numbers, often referred to as chromosomes, which are iteratively modified to obtain a solution better adapted to the problem at hand. This is why they are called genetic algorithms.

This algorithm is designed and built following these steps:

Objective Function: Define the problem and establish the objective function that leads to the desired solution.
Population: Create an initial population of potential solutions and represent them in binary form (chromosomes).
Crossover: Combine pairs of chromosomes to create offspring or new sets of potential solutions.
Mutation: Modify the genes of each chromosome to increase diversity and avoid suboptimal solutions.
Evaluation: Convert the variables of all chromosomes from binary to real numerical format and use them to evaluate the objective function.
Selection: Use statistical models to select a set of solutions that form a new population.
Repeat: Repeat steps 2 to 7 with the new population until a convergence criterion is met.

Genetic Algorithm Flowchart. Image created by the author.

Finally, we present some examples that will enhance your overall comprehension of genetic algorithms.

Optimization

Genetic algorithms can be implemented to optimize problems within certain constraints and specific conditions. They are applicable in various industries, including engineering, data science, and business planning.

For example, we have utilized this algorithm in a scientific research to optimize complex mathematical models. Using a genetic algorithm significantly accelerated this process, which would otherwise be time-consuming. Moreover, its reproducibility becomes possible for other authors since it establishes a standardized approach. Another example is the optimization of airline expenses, achieved by efficiently managing resources based on factors such as passenger count, route, and aircraft type.

In general, utilizing optimization algorithms enables the maximization of benefits, improvement of efficiency, and minimization of costs.

Hyperparameter Tuning

When building machine learning models, it is crucial to select optimal hyperparameters. For this, hyperparameter optimization plays a fundamental role in achieving high-performance and generalizable ML models. It allows users to fit the ML model to the characteristics of the problem and the data, avoid overfitting or underfitting, improve computational efficiency, and obtain consistent and reproducible results.