Particle swarm optimization using Python – News Couple
ANALYTICS

Particle swarm optimization using Python


This article was published as part of the Data Science Blogathon.

an introduction

There are many approaches one can take to either reduce or enlarge any function so that the optimal value can be found. You can find many optimization solutions on the internet but in the end, no one solution is best for all. Everyone has their own benefits and drawbacks. What we will discuss here is PSO or particle swarm optimization. You can also say that it is an area of ​​improvement that belongs to the nature-inspired method of computing. It is a kind of algorithm that searches for the optimal solution in a specific space in a direct way.

The solution works with logic where a filter is worked out to make it better by a certain measure of quality and then the process is repeated iteratively until no better solution is available and thus the same optimal state is reached. The language here will be Python and we will see a practical implementation of it using the “PySwarms” Python package. We will cover the following topics here:

  1. PSO: Particle Swarm Improvement
  2. internal business
  3. Variables or types of PSO
  4. Implementing a PSO with PySwarms

What is particle swarm optimization (PSO)?

In the nineties there were a number of studies conducted on the topic of social behavior of animal populations. These research studies concluded that one of the lessons learned is that some animal groups such as birds or fish are able to pass on their knowledge or communicate with others in the group. This sometimes helped them survive attacks or gain an advantage over other animal groups when evading predators. Based on all this research, Kennedy and Eberhardt proposed the PSO algorithm. In 1995, this PSO algorithm was proposed as a metaheuristic algorithm. This algorithm has optimized continuous nonlinear functions based on the concept of flock intelligence displayed in herds and shoals.

What is particle swarm optimization
picture 1

The studies concluded that there was a sharing of knowledge or wisdom among the animals which helped them gain insight into a situation they had not encountered before. In simpler words, finding food or escaping from prey becomes easy when all the birds of the flock share their observations and knowledge. Thus it becomes easy to select the optimal solution intelligently for the swarm.

The inner workings of improving particle swarm

Swarm behavior can be of two types. The first is the exploratory behavior where the animals search for their objects in a larger solution space. The second is exploitative behavior where the swarm’s intelligence helps them search for a smaller, more ideal area. PSOs and their standards are often designed to find the optimal balance between exploration and exploitation. This ensures a good convergence rate to the optimum level.

2.1 Affinity Particle Swarm Optimization

In PSO convergence it does not matter how a swarm will work but convergence to a specific local best when the different personal best things let’s say P or the most well-known position of the swarm or also known as G reach the local optimal problem. The ability that a PSO algorithm has that it can exploit and explore can be affected by the structure of the structure. So for different architectures, the algorithm can converge differently due to the shape of the structure which is generally either a local loop or a global star. The structure of the structure determines factors such as search, information sharing, velocity, and even the direction each particle should follow.

The inner workings of improving particle swarm
picture 2

A PSO having a global stellar structure where all particles are related to each other has the advantage of one of the shortest average distance but a local ring structure where one particle is bound to the two nearest particles has the highest average distance in the entire swarm as shown in the image above. The figure on the left is a global star structure while the figure on the left is the local ring structure. Each group consists of 16 particles and you can see why the average distance for both cases is different.

2.2 Adaptation Mechanism

In the adaptive mechanism, one can implement it with the desired trade-off between convergence and divergence or exploitation and exploration. APSO optimization or adaptive particle swarm can outperform normal PSO performance. With a faster convergence time, APSO can also perform any global search across the entire search space.

Particle swarm optimization variables

PSO algorithms can be of different types, even simple ones. Particles and velocities can be initiated in different ways. Update the Swarm and then set the values ​​for Pi, G, etc.

3.1 Improved gradient particle swarm

We can create a gradient-based PSO by combining the efficiency of a PSO in exploring many local minima with gradient-based local search algorithms. This helps us to accurately calculate the local minimum. The PSO algorithm can be used in gradient-based PSO also where many local minima are detected along with a deep gravitational sink for a deep local minima. The deep minimum can then be confirmed using local gradient-based search techniques.

Gradient particle swarm optimization
Picture 3: https://www.hindawi.com/journals/ddns/2021/8378579/fig1/

3.2 Hybrid particle swarm optimization

In order to further and make the improvement process better, newer and more advanced types of PSO forms are being tested and used and are an ongoing area of ​​study. Hybrid PSO is where regular PSO is combined with other optimization technology that helps to improve it. One example is biogeography-based optimization with the implementation of an effective learning mechanism that will improve and enhance the existing PSO.

Hybrid particle swarm optimization
Picture 4

4. Particle swarm optimization application using PySwarms

As the name suggests, PySwarms is a Python-based tool that helps in swarm optimization. This tool is used by researchers, practitioners, and students alike in order to implement the PSO algorithm using a high-level interface. PySwarms is the best tool to combine swarm optimization with core optimization.

This tool allows you to implement and use a number of multi-particle crowd optimization techniques. It is also very easy to use and adaptable to different projects. An optimization problem can also benefit from support modules.

We will now move on to using the global best optimizer using the functional PySwarm API which is “pyswarms.single.GBestPSOWe will draw it in 2D and 3D.

You can install PySwarm with the following commands:

!pip install pyswarms

Now we’ll start by importing some modules first:

Code:

# Import all the modules
import numpy as np
# Importing PySwarms
import pyswarms as ps
from pyswarms.utils.functions import single_obj as fx
import matplotlib.pyplot as plt
from pyswarms.utils.plotters import plot_contour, plot_surface
from pyswarms.utils.plotters.formatters import Designer
from pyswarms.utils.plotters.formatters import Mesher

After importing all of them we will start improving our sphere function, you can put any random settings to start in our optimizer. The three main steps here are:

  1. 1. Set the hyperparameters to configure the swarm as a dictionary
  2. 2. To create an instance of the optimizer, pass the dictionary with all relevant input parameters
  3. 3. The best cost and position can be provided in a variable by calling”improvement () ” Job.

Code:

# Set-up all the hyperparameters
options = 'c1': 0.5, 'c2': 0.3, 'w':0.9
# Call an instance of PSO
optimizer = ps.single.GlobalBestPSO(n_particles=10, dimensions=2, options=options)
# Perform the optimization
cost, pos = optimizer.optimize(fx.sphere, iters=1000)

Produce :

2021-10-31 15:38:44,604 - pyswarms.single.global_best - INFO - Optimize for 1000 iters with 'c1': 0.5, 'c2': 0.3, 'w': 0.9
pyswarms.single.global_best: 100%|██████████|1000/1000, best_cost=4.56e-42
2021-10-31 15:38:46,880 - pyswarms.single.global_best - INFO - Optimization finished | best cost: 4.556071389349258e-42, best pos: [ 6.91683472e-22 -2.01931804e-21]

4.1 Now in the job visualization

PySwarm already comes with several tools that will help you visualize swarm behavior. These behaviors are generated on top of matplotlib resulting in highly customizable graphs that are also easy to use. There are also modes of animation that are “plot_contour()” And “plot_surface()methods. These methods plot the particle in 2D and 3D space respectively. To plot the sphere function, we need to add a specific function called the lattice function to our swarm.

These help to visualize the particles graphically. We have already imported the “Mesher” class which helps us to achieve this. The “pyswarms.utils.plotters.formattersIt has many different formats so that you can customize the plots. Apart from Mesher, there is a ‘Designer’ class which modifies shape size, font size and more as well as Animation class to set repetitions and animation delay.

2D plot code:

m = Mesher(func=fx.sphere)
animation = plot_contour(pos_history=optimizer.pos_history,
                         mesher=m,
                         mark=(0,0))

Produce :

#Code for 3D Plot :
# The preprocessing
pos_history_3d = m.compute_history_3d(optimizer.pos_history)
# Adjusting the figure
d = Designer(limits=[(-1,1), (-1,1), (-0.1,1)], label=['x-axis', 'y-axis', 'z-axis'])
animation3d = plot_surface(pos_history=pos_history_3d, # The cost_history that we computed
                           mesher=m, designer=d,       # Various Customizations
                           mark=(0,0,0))               # Mark the minima

Produce :

Particle swarm optimization - cover page

Note: .gif support will be added soon so it will be uploaded to YouTube for now. The first 10 seconds of the video are essential, the rest not so much. The collab notebook link is given below:

https://colab.research.google.com/drive/1JO5p-wlC6F2g3hsH2lCx3mvHjjBSswuf?usp=sharing

conclusion

So in this post, we saw the theory and how PSO works and explored the inner mechanism. We have seen some variants of it and how it has been used in various fields and societies. Last but not least, we also made our own mini python based swarm. Feel free to message me if you run into any issues or encounter any issues.

Ankita Roy

Information Technology Class of 2022 is open for full time / internships

[email protected]

Image links:

  1. Picture 1 – https://www.hindawi.com/journals/ddns/2021/8378579/fig1/
  2. Picture 2 – https://www.hindawi.com/journals/ddns/2021/8378579/fig2/
  3. Picture 3 – https://www.hindawi.com/journals/ddns/2021/8378579/fig1/
  4. Picture 4: https://www.hindawi.com/journals/ddns/2021/8378579/fig5/
  5. https://unsplash.com/photos/b7MZ6iGIoSI
  6. https://unsplash.com/photos/qvIeCwqxzU4

The media described in this article is not owned by Analytics Vidhya and is used at the author’s discretion.



Source link

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button