# Particle swarm optimization and gravity search algorithm for solving MLP problem matlab source code

## 1. Gravity Gravitation Search Algorithms

### 1.1 Particle Swarm Algorithm

Particle Swarm Optimization (PSO) was first proposed by Eberhart and Kennedy in 1995. Its basic concept originates from the study of bird foraging behavior.Imagine a scene where a group of birds are randomly searching for food. There is only one piece of food in this area. All the birds don't know where the food is, but they know how far away it is from food.The simplest and most effective strategy?Search for the nearest individual to food in the bird population.The PSO algorithm derives inspiration from the behavioral characteristics of this biological population and is used to solve optimization problems.

A particle is used to simulate the individual bird mentioned above. Each particle can be considered as a searching individual in the N-dimensional search space. The current position of the particle is a candidate solution to the corresponding optimization problem.A particle's flight process is the individual's search process. The particle's flight speed can be dynamically adjusted according to the historical optimal position of the particle and the historical optimal position of the population. A particle has only two attributes: speed and position, speed represents the speed of movement, and position represents the direction of movement.The optimal solution for each individual particle to search for is called the individual extreme value, and the optimal individual extreme value in the particle swarm is the current global optimal solution.Continuous iteration to update speed and location.Finally, the optimal solution satisfying the termination condition is obtained.

The algorithm flow is as follows:

1. Initialization

First, we set the maximum number of iterations, the number of independent variables of the objective function, the maximum speed of particles, and the location information for the entire search space. We randomly initialize the speed and location in the speed interval and search space, set the particle size to M, and randomly initialize a flight speed for each particle.

2. Individual Extremum and Global Optimal Solution

Define the fitness function, and the individual extreme values are the best solutions found for each particle. From these optimal solutions, a global value is found, which is called this global optimal solution.Update compared to historical global optimum.

3. Update speed and position formulas

4. Termination Conditions

(1) reach the set number of iterations; (2) The difference between algebras satisfies the minimum bound

This is the most basic standard PSO algorithm flow.As with other swarm intelligence algorithms, there is always a conflict between the diversity of the population and the convergence speed of the PSO algorithm in the optimization process. The improvement of standard PSO algorithm, whether it is the selection of parameters, the adoption of niche technology or the combination of other techniques and PSO, aims at maintaining the diversity of the population while strengthening the local search ability of the algorithm.Prevents premature convergence while fast convergence occurs.

### 1.2 Gravity Search Algorithm

The Gravitational Search Algorithm (GSA) is a random heuristic search algorithm proposed by Esmat Rashedi et al in 2009. It is inspired by Newton's laws of gravity and motion: 1. Any two particles are attracted to each other through the direction of the center line.The magnitude of gravitation is proportional to the product of their mass and inversely proportional to the square of their distance.2. Force accelerates the body.

In GSA, a particle is abstracted as a solution in the solution space, and there is a mutual attraction between the solutions, which is determined by the quality of the solution and the distance between the two solutions. The quality of the particle is abstracted as the value of the evaluation function of the solution.In solution space, each solution is accelerated by the attraction of other solutions, and the acceleration provided by a larger mass (better value of the evaluation function) is greater, thus moving the solution in the direction of the better solution.

Seeing this, some little partners will think that GSA is an algorithm similar to PSO. Yes, GSA is consistent with PSO's outer framework, but their most critical particle movement strategies are different:

1. In PSO, only two optimal positions, pbest and gbest, are used for particle movement.However, in GSA, the direction of the agent is calculated based on the sum of effort obtained by all other agents or some of the better agents.
2.PSO uses a memory to update speed (due to pbest and gbest).However, GSA is memory-free, and only the current location of the agent will play a role in the update process.
3. In PSO, the update does not take into account the distance between solutions, whereas in GSA, the force is inversely proportional to the distance between solutions.
4.PSO simulates the social behavior of birds, and GSA is inspired by physical phenomena.

The main processes of GSA are as follows:

1. Determine the search space.
2. Initialize the individual population randomly.
3. Assess the population suitability.
4. Update the gravitational constant G, update the best individual best and worst individual in the population, and update the quality of the individual.
5. Calculate the total gravitational pull of each individual in different directions.
6. Calculate the individual's acceleration based on the speed at which the individual is updated.
7. Update the individual's position in the solution space according to the speed.
8. Repeat steps 3-7 until the stop standard is reached.

Algorithmic flow:

## 2. Partial Codes



%%%%%  Multi-layer Perceptron (MLP) Training using CPSOGSA %%%%%
%
% Citation
% Rather, S.A. and Bala, P.S. (2020), "A hybrid constriction coefficient-based particle swarm optimization and gravitational search algorithm for training multi-layer perceptron",
% International Journal of Intelligent Computing and Cybernetics, Vol. 13 No. 2, pp. 129-165. https://doi.org/10.1108/IJICC-09-2019-0105
%
%  Developed in MATLAB R2013b                                       %
%

clear all
close all
clc

Q=1;            % ACO Parameter
tau0=10;        % Initial Phromone             (ACO)
alpha=0.3;      % Phromone Exponential Weight  (ACO)
rho=0.1;        % Evaporation Rate             (ACO)
beta_min=0.2;   % Lower Bound of Scaling Factor (DE)
beta_max=0.8;   % Upper Bound of Scaling Factor (DE)
pCR=0.2;        % Crossover Probability         (DE)
Runno=10;

SearchAgents_no=20; % Number of search agents

% classification datasets

Function_name='F1'; %MLP_XOR dataset
%  Function_name='F2'; %MLP_Baloon dataset
%  Function_name='F3'; %MLP_Iris dataset
% Function_name='F4'; %MLP_Cancer dataset
% Function_name='F5'; %MLP_Heart dataset

% Function approximation datasets

% Function_name='F6'; %MLP_Sigmoid dataset
% Function_name='F7'; %MLP_Cosine dataset
% Function_name='F8'; %MLP_Sine dataset
% Function_name='F9'; %MLP_Sphere dataset

% Load details of the selected data set
[lb,ub,dim,fobj]=Get_Functions_details(Function_name);

ElitistCheck=1;
min_flag=1;
Rpower=1;
Max_iteration=500; % Maximum numbef of iterations

%
if Function_name=='F1'
input=  [0 0 0 0 1 1 1 1;0 0 1 1 0 0 1 1;0 1 0 1 0 1 0 1];
target3=[0 1 1 0 1 0 0 1];
Hno=7;
dim = 5*7+1;                      % Dimension of the problem

for i=1:1:Runno

Rrate=0;
[Fbest,Lbest,BestChart]=GSA(SearchAgents_no,Max_iteration,ElitistCheck,min_flag,Rpower,lb,ub,dim,fobj);
BestSolutions1(i) = Fbest;
[gBestScore,gBest,GlobalBestCost]= CPSOGSA(SearchAgents_no,Max_iteration,lb,ub,dim,fobj);
BestSolutions4(i) = gBestScore;
[BestSolACO,BestAnt,BestCostACO] = ACO(SearchAgents_no, Max_iteration,Q,tau0,alpha,rho,lb,ub,dim,fobj);
BestSolutions5(i) = BestSolACO.Cost;
[BestCost,Best_Hab,BestSol] = bbo( SearchAgents_no, Max_iteration,lb,ub,dim,fobj);
BestSolutions6(i) = BestSol.Cost;
[BestSolDE,DBestSol,BestCostDE] = DE(SearchAgents_no, Max_iteration,beta_min,beta_max,pCR,lb,ub,dim,fobj);
BestSolutions7(i) = BestSolDE.Cost ;

W=Lbest(1:4*Hno);
B=Lbest(4*Hno+1:dim);
W=gBest(1:4*Hno);
B=gBest(4*Hno+1:dim);
W=BestAnt(1:4*Hno);
B=BestAnt(4*Hno+1:dim);
W=Best_Hab(1:4*Hno);
B=Best_Hab(4*Hno+1:dim);
W=DBestSol(1:4*Hno);
B=DBestSol(4*Hno+1:dim);

for pp=1:8
actualvalue=my_simulate(3,Hno,1,W,B,input(:,pp)');
if(target3(pp)==1)
if (actualvalue>=0.95)
Rrate=Rrate+1;
end
end
if(target3(pp)==0)
if (actualvalue(1)<0.05)
Rrate=Rrate+1;
end
end
end

Classification_rate(i)=(Rrate/8)*100;

disp(['GSA is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(Fbest),' Mean_Classification rate = ' , num2str(Classification_rate(i)),')'])
% disp(['CPSOGSA is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(gBestScore),' Mean_Classification rate = ' , num2str(Classification_rate(i)),')'])
% disp(['ACO is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(BestSolACO.Cost),' Mean_Classification rate =' , num2str(Classification_rate(i)),')'])
% disp(['BBO is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(BestSol.Cost),' Mean_Classification rate = ' , num2str(Classification_rate(i)),')'])
% disp(['DE is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(BestSolDE.Cost),' Mean_Classification rate = ' , num2str(Classification_rate(i)),')'])
end

A_Classification_rate=mean(Classification_rate);
Average= mean(BestSolutions1);
StandDP=std(BestSolutions1);
Med = median(BestSolutions1);
[BestValueP I] = min(BestSolutions1);
[WorstValueP IM]=max(BestSolutions1);
end

%
% if Function_name=='F2'
%
%  x=sortrows(baloon,2);
%  %I2=x(1:150,1:4);
%  I2(:,1)=x(1:20,1);
%  I2(:,2)=x(1:20,2);
%  I2(:,3)=x(1:20,3);
%  I2(:,4)=x(1:20,4);
%  T=x(1:20,5);
%
%
% Hno=9;
% dim = 6*9+1;
%
%    for i=1:1:Runno
%         [Fbest,Lbest,BestChart]=GSA(SearchAgents_no,Max_iteration,ElitistCheck,min_flag,Rpower,lb,ub,dim,fobj);
%         BestSolutions1(i) = Fbest;
%          [gBestScore,gBest,GlobalBestCost]= CPSOGSA(SearchAgents_no,Max_iteration,lb,ub,dim,fobj);
%          BestSolutions4(i) = gBestScore;
%         [BestSolACO,BestAnt,BestCostACO] = ACO(SearchAgents_no, Max_iteration,Q,tau0,alpha,rho,lb,ub,dim,fobj);
%         BestSolutions5(i) = BestSolACO.Cost;
%         [BestCost,Best_Hab,BestSol] = bbo( SearchAgents_no, Max_iteration,lb,ub,dim,fobj);
%         BestSolutions6(i) = BestSol.Cost;
%         [BestSolDE,DBestSol,BestCostDE] = DE(SearchAgents_no, Max_iteration,beta_min,beta_max,pCR,lb,ub,dim,fobj);
%         BestSolutions7(i) = BestSolDE.Cost ;
%         Rrate=0;

%W=Lbest(1:45);
%B=Lbest(46:55);
%W=gBest(1:45);
%B=gBest(46:55);
%W=BestAnt(1:45);
% B=BestAnt(46:55);
% W=Best_Hab(1:45);
% B=Best_Hab(46:55);
% W=DBestSol(1:45);
% B=DBestSol(46:55);

%         for pp=1:20
%             actualvalue=my_simulate(4,9,1,W,B,I2(pp,:));
%             if(T(pp)==1)
%                 if (actualvalue>=0.95)
%                     Rrate=Rrate+1;
%                 end
%             end
%             if(T(pp)==0)
%                 if (actualvalue(1)<0.05)
%                     Rrate=Rrate+1;
%                 end
%             end
%
%         end
% end
% end
% % %
%  if Function_name=='F3'
%
%  x=sortrows(iris,2);
%  I2=x(1:150,1:4);
%  H2=x(1:150,1);
%  H3=x(1:150,2);
%  H4=x(1:150,3);
%  H5=x(1:150,4);
%  T=x(1:150,5);
%  I=(I2-0.1)./(7.9-0.1);
%  H2=H2';
%  [xf,PS] = mapminmax(H2);
%  I2(:,1)=xf;
%
%  H3=H3';
%  [xf,PS2] = mapminmax(H3);
%  I2(:,2)=xf;
%
%  H4=H4';
%  [xf,PS3] = mapminmax(H4);
%  I2(:,3)=xf;
%
%  H5=H5';
%  [xf,PS4] = mapminmax(H5);
%  I2(:,4)=xf;
%  Thelp=T;
%  T=T';
%  [yf,PS5]= mapminmax(T);
%  T=yf;
%  T=T';
%
%     for i=1:1:Runno
%         [Fbest,Lbest,BestChart]=GSA(SearchAgents_no,Max_iteration,ElitistCheck,min_flag,Rpower,lb,ub,dim,fobj);
%         BestSolutions1(i) = Fbest;
%          [gBestScore,gBest,GlobalBestCost]= CPSOGSA(SearchAgents_no,Max_iteration,lb,ub,dim,fobj);
%          BestSolutions4(i) = gBestScore;
%         [BestSolACO,BestAnt,BestCostACO] = ACO(SearchAgents_no, Max_iteration,Q,tau0,alpha,rho,lb,ub,dim,fobj);
%         BestSolutions5(i) = BestSolACO.Cost;
%         [BestCost,Best_Hab,BestSol] = bbo( SearchAgents_no, Max_iteration,lb,ub,dim,fobj);
%         BestSolutions6(i) = BestSol.Cost;
%         [BestSolDE,DBestSol,BestCostDE] = DE(SearchAgents_no, Max_iteration,beta_min,beta_max,pCR,lb,ub,dim,fobj);
%         BestSolutions7(i) = BestSolDE.Cost ;

%         Rrate=0;
%             W=Lbest(1:63);
%             B=Lbest(64:75);
%             W=gBest(1:63);
%             B=gBest(64:75);
%             W=BestAnt(1:63);
%             B=BestAnt(64:75);
%             W=Best_Hab(1:63);
%             B=Best_Hab(64:75);
%             W=DBestSol(1:63);
%             B=DBestSol(64:75);

%         for pp=1:150
%             actualvalue=my_simulate(4,9,3,W,B,I2(pp,:));
%             if(T(pp)==-1)
%                 if (actualvalue(1)>=0.95 && actualvalue(2)<0.05 && actualvalue(3)<0.05)
%                     Rrate=Rrate+1;
%                 end
%             end
%             if(T(pp)==0)
%                 if (actualvalue(1)<0.05 && actualvalue(2)>=0.95 && actualvalue(3)<0.05)
%                     Rrate=Rrate+1;
%                 end
%             end
%             if(T(pp)==1)
%                 if (actualvalue(1)<0.05 && actualvalue(2)<0.05 && actualvalue(3)>=0.95)
%                     Rrate=Rrate+1;
%                 end
%             end
%         end
% end
% end
%
% if Function_name=='F4'
%
%  x=Cancer;
%  %I2=x(1:150,1:4);
%  H2=x(1:699,2:11);
%  for iii=1:699
%      for jjj=1:10
%          H2(iii,jjj)=((H2(iii,jjj)-1)/9);
%      end
%  end
%  I2=H2(1:699,1:9);
%
%  T=H2(1:699,10);
%  Hno=19;
%  dim=11*19;
%
%     for i=1:1:Runno
%         Rrate=0;
%         [Fbest,Lbest,BestChart]=GSA(SearchAgents_no,Max_iteration,ElitistCheck,min_flag,Rpower,lb,ub,dim,fobj);
%         BestSolutions1(i) = Fbest;
%          [gBestScore,gBest,GlobalBestCost]= CPSOGSA(SearchAgents_no,Max_iteration,lb,ub,dim,fobj);
%          BestSolutions4(i) = gBestScore;
%         [BestSolACO,BestAnt,BestCostACO] = ACO(SearchAgents_no, Max_iteration,Q,tau0,alpha,rho,lb,ub,dim,fobj);
%         BestSolutions5(i) = BestSolACO.Cost;
%         [BestCost,Best_Hab,BestSol] = bbo( SearchAgents_no, Max_iteration,lb,ub,dim,fobj);
%         BestSolutions6(i) = BestSol.Cost;
%         [BestSolDE,DBestSol,BestCostDE] = DE(SearchAgents_no, Max_iteration,beta_min,beta_max,pCR,lb,ub,dim,fobj);
%         BestSolutions7(i) = BestSolDE.Cost ;

% W=Lbest(1:10*Hno);
% B=Lbest(10*Hno+1:dim);
% W=gBest(1:10*Hno);
% B=gBest(10*Hno+1:dim);
% W=BestAnt(1:10*Hno);
% B=BestAnt(10*Hno+1:dim);
% W=Best_Hab(1:10*Hno);
% B=Best_Hab(10*Hno+1:dim);
% W=DBestSol(1:10*Hno);
% B=DBestSol(10*Hno+1:dim);

%         for pp=600:699
%             actualvalue=my_simulate(9,Hno,1,W,B,I2(pp,:) );
%             if(T(pp)>=0.3 && T(pp)<0.4)
%                 if (abs(actualvalue-0.333333333333333)<0.1)
%                     Rrate=Rrate+1;
%                 end
%             end
%             if(T(pp)>=0.1 && T(pp)<0.2)
%                 if (abs(actualvalue-0.111111111111111)<0.1)
%                     Rrate=Rrate+1;
%                 end
%             end
%
%         end
%   end
% end

% if Function_name=='F5'
%
%  x=Heart;
% % I2=x(1:150,1:4);
%  I2(:,1)=x(1:80,2);
%  I2(:,2)=x(1:80,3);
%  I2(:,3)=x(1:80,4);
%  I2(:,4)=x(1:80,5);
%  I2(:,5)=x(1:80,6);
%  I2(:,6)=x(1:80,7);
%  I2(:,7)=x(1:80,8);
%  I2(:,8)=x(1:80,9);
%  I2(:,9)=x(1:80,10);
%  I2(:,10)=x(1:80,11);
%  I2(:,11)=x(1:80,12);
%  I2(:,12)=x(1:80,13);
%  I2(:,13)=x(1:80,14);
%  I2(:,14)=x(1:80,15);
%  I2(:,15)=x(1:80,16);
%  I2(:,16)=x(1:80,17);
%  I2(:,17)=x(1:80,18);
%  I2(:,18)=x(1:80,19);
%  I2(:,19)=x(1:80,20);
%  I2(:,20)=x(1:80,21);
%  I2(:,21)=x(1:80,22);
%  I2(:,22)=x(1:80,23);
%  T=x(1:80,1);
%
%  Hno=45;
% dim = 24*45+1;
%
%     for i=1:1:Runno
%
%         Rrate=0;
%         [Fbest,Lbest,BestChart]=GSA(SearchAgents_no,Max_iteration,ElitistCheck,min_flag,Rpower,lb,ub,dim,fobj);
%         BestSolutions1(i) = Fbest;
%         [gBestScore,gBest,GlobalBestCost]= CPSOGSA(SearchAgents_no,Max_iteration,lb,ub,dim,fobj);
%         BestSolutions4(i) = gBestScore;
%         [BestSolACO,BestAnt,BestCostACO] = ACO(SearchAgents_no, Max_iteration,Q,tau0,alpha,rho,lb,ub,dim,fobj);
%         BestSolutions5(i) = BestSolACO.Cost;
%         [BestCost,Best_Hab,BestSol] = bbo( SearchAgents_no, Max_iteration,lb,ub,dim,fobj);
%         BestSolutions6(i) = BestSol.Cost;
%         [BestSolDE,DBestSol,BestCostDE] = DE(SearchAgents_no, Max_iteration,beta_min,beta_max,pCR,lb,ub,dim,fobj);
%         BestSolutions7(i) = BestSolDE.Cost ;

% W=Lbest(1:23*Hno);
% B=Lbest(23*Hno+1:dim);
% W=gBest(1:23*Hno);
% B=gBest(23*Hno+1:dim);
% W=BestAnt(1:23*Hno);
% B=BestAnt(23*Hno+1:dim);
% W=Best_Hab(1:23*Hno);
% B=Best_Hab(23*Hno+1:dim);
% W=DBestSol(1:23*Hno);
% B=DBestSol(23*Hno+1:dim);

%         for pp=1:80
%             actualvalue=my_simulate(22,Hno,1,W,B,I2(pp,:) );
%             if(T(pp)==1)
%                 if (actualvalue>=0.95)
%                     Rrate=Rrate+1;
%                 end
%             end
%             if(T(pp)==0)
%                 if (actualvalue(1)<0.05)
%                     Rrate=Rrate+1;
%                 end
%             end
%
%         end
%      end
% end

% if Function_name=='F6'  %% Sigmoid
%
%     Hnode=15;
% dim = 3*Hnode+1;
%
% %for test 3 times more than the training samples
% %   xf1=[0:0.01:pi];
% %   yf1=sin(2.*xf1);
% %   yf1=yf1.*exp(-xf1);
%  xf1=[-3:0.05:3];
% %  yf1=sin(2.*xf1);
% %  yf1=yf1.*exp(-xf1);
%  %yf1=xf1.^2;
% %yf1=xf1.^4-6.*xf1.^2+3;
% yf1=sigmf(xf1,[1 0]);
%
% %   xf1=[-2*pi:0.05:2*pi];
% %   yf1=sin(2.*xf1);
%   %yf1=yf1.*exp(-xf1);
%  yNN=zeros(1,10);
%  [xf,PS] = mapminmax(xf1);
%  [yf,PS2]= mapminmax(yf1);
%   [M N]=size(xf);
%   test_error=zeros(1,Runno);
%     for i=1:1:Runno
%         [Fbest,Lbest,BestChart]=GSA(SearchAgents_no,Max_iteration,ElitistCheck,min_flag,Rpower,lb,ub,dim,fobj);
%         BestSolutions1(i) = Fbest;
%          [gBestScore,gBest,GlobalBestCost]= CPSOGSA(SearchAgents_no,Max_iteration,lb,ub,dim,fobj);
%          BestSolutions4(i) = gBestScore;
%         [BestSolACO,BestAnt,BestCostACO] = ACO(SearchAgents_no, Max_iteration,Q,tau0,alpha,rho,lb,ub,dim,fobj);
%         BestSolutions5(i) = BestSolACO.Cost;
%         [BestCost,Best_Hab,BestSol] = bbo( SearchAgents_no, Max_iteration,lb,ub,dim,fobj);
%         BestSolutions6(i) = BestSol.Cost;
%         [BestSolDE,DBestSol,BestCostDE] = DE(SearchAgents_no, Max_iteration,beta_min,beta_max,pCR,lb,ub,dim,fobj);
%         BestSolutions7(i) = BestSolDE.Cost ;

% W=1:2*Hnode;
% B=2*Hnode+1:3*Hnode+1;
% W=gBest(1:2*Hnode);
% B=gBest(2*Hnode+1:3*Hnode+1);
% W=BestAnt(1:2*Hnode);
% B=BestAnt(2*Hnode+1:3*Hnode+1);
% W=Best_Hab(1:2*Hnode);
% B=Best_Hab(2*Hnode+1:3*Hnode+1);
% W=DBestSol(1:2*Hnode);
% B=DBestSol(2*Hnode+1:3*Hnode+1);
%
%         for pp=1:N
%             yNN(pp)=my_simulate(1,15,1, W,B,xf(pp));
%         end
%
% disp(['GSA is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(Fbest),')'])
% disp(['CPSOGSA is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(gBestScore), ')'])
% disp(['ACO is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(BestSolACO.Cost),')'])
% disp(['BBO is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(BestSol.Cost),')'])
% disp(['DE is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(BestSolDE.Cost),')'])
%     end
% end

% if Function_name=='F7' %% Cosine
%
% Hnode=15;
% dim = 3*Hnode+1;
%
% %for test 3 times more than the training samples
% %   xf1=[0:0.01:pi];
% %   yf1=sin(2.*xf1);
% %   yf1=yf1.*exp(-xf1);
%  %xf1=[-3:0.05:3];
% %  yf1=sin(2.*xf1);
% %  yf1=yf1.*exp(-xf1);
%  %yf1=xf1.^2;
% %yf1=xf1.^4-6.*xf1.^2+3;
%
%   xf1=[1.25:0.04:2.75];
%   yf1=power(cos(xf1.*pi/2),7);
%   %yf1=yf1.*exp(-xf1);
%  yNN=zeros(1,10);
%  [xf,PS] = mapminmax(xf1);
%  [yf,PS2]= mapminmax(yf1);
%   [M N]=size(xf);
%   test_error=zeros(1,Runno);
%     for i=1:1:Runno
%         [Fbest,Lbest,BestChart]=GSA(SearchAgents_no,Max_iteration,ElitistCheck,min_flag,Rpower,lb,ub,dim,fobj);
%         BestSolutions1(i) = Fbest;
%         [gBestScore,gBest,GlobalBestCost]= CPSOGSA(SearchAgents_no,Max_iteration,lb,ub,dim,fobj);
%         BestSolutions4(i) = gBestScore;
%         [BestSolACO,BestAnt,BestCostACO] = ACO(SearchAgents_no, Max_iteration,Q,tau0,alpha,rho,lb,ub,dim,fobj);
%         BestSolutions5(i) = BestSolACO.Cost;
%         [BestCost,Best_Hab,BestSol] = bbo( SearchAgents_no, Max_iteration,lb,ub,dim,fobj);
%         BestSolutions6(i) = BestSol.Cost;
%         [BestSolDE,DBestSol,BestCostDE] = DE(SearchAgents_no, Max_iteration,beta_min,beta_max,pCR,lb,ub,dim,fobj);
%         BestSolutions7(i) = BestSolDE.Cost ;
%
% W= Lbest(1:2*Hnode);
% B=Lbest(2*Hnode+1:3*Hnode+1);
% W=gBest(1:2*Hnode);
% B=gBest(2*Hnode+1:3*Hnode+1);
% W=BestAnt(1:2*Hnode);
% B=BestAnt(2*Hnode+1:3*Hnode+1);
% W=Best_Hab(1:2*Hnode);
% B=Best_Hab(2*Hnode+1:3*Hnode+1);
% W=DBestSol(1:2*Hnode);
% B=DBestSol(2*Hnode+1:3*Hnode+1);

%         for pp=1:N
%             yNN(pp)=my_simulate(1,15,1, W,B,xf(pp));
%         end
% %
% disp(['GSA is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(Fbest)
% disp(['CPSOGSA is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(gBestScore), ')'])
% disp(['ACO is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(BestSolACO.Cost),')'])
% disp(['BBO is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(BestSol.Cost),')'])
% disp(['DE is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(BestSolDE.Cost),')'])
%     end
% end

% if Function_name=='F8' %%Sine
%
% Hnode=15;
% dim = 3*Hnode+1;
%
% %for test 3 times more than the training samples
% %   xf1=[0:0.01:pi];
% %   yf1=sin(2.*xf1);
% %   yf1=yf1.*exp(-xf1);
%  %xf1=[-3:0.05:3];
% %  yf1=sin(2.*xf1);
% %  yf1=yf1.*exp(-xf1);
%  %yf1=xf1.^2;
% %yf1=xf1.^4-6.*xf1.^2+3;
%
%   xf1=[-2*pi:0.05:2*pi];
%   yf1=sin(2.*xf1);
%   %yf1=yf1.*exp(-xf1);
%  yNN=zeros(1,10);
%  [xf,PS] = mapminmax(xf1);
%  [yf,PS2]= mapminmax(yf1);
%   [M N]=size(xf);
%   test_error=zeros(1,Runno);
%     for i=1:1:Runno
% %         [Fbest,Lbest,BestChart]=GSA(SearchAgents_no,Max_iteration,ElitistCheck,min_flag,Rpower,lb,ub,dim,fobj);
% %         BestSolutions1(i) = Fbest;
% %         [gBestScore,gBest,GlobalBestCost]= CPSOGSA(SearchAgents_no,Max_iteration,lb,ub,dim,fobj);
% %         BestSolutions4(i) = gBestScore;
% %         [BestSolACO,BestAnt,BestCostACO] = ACO(SearchAgents_no, Max_iteration,Q,tau0,alpha,rho,lb,ub,dim,fobj);
% %         BestSolutions5(i) = BestSolACO.Cost;
% %         [BestCost,Best_Hab,BestSol] = bbo( SearchAgents_no, Max_iteration,lb,ub,dim,fobj);
% %         BestSolutions6(i) = BestSol.Cost;
%         [BestSolDE,DBestSol,BestCostDE] = DE(SearchAgents_no, Max_iteration,beta_min,beta_max,pCR,lb,ub,dim,fobj);
%         BestSolutions(i) = BestSolDE.Cost ;
% %
% W=Lbest(1:2*Hnode);
% B= Lbest(2*Hnode+1:3*Hnode+1);
% W=gBest(1:2*Hnode);
% B=gBest(2*Hnode+1:3*Hnode+1);
% W=BestAnt(1:2*Hnode);
% B=BestAnt(2*Hnode+1:3*Hnode+1);
% W=Best_Hab(1:2*Hnode);
% B=Best_Hab(2*Hnode+1:3*Hnode+1);
% W=DBestSol(1:2*Hnode);
% B=DBestSol(2*Hnode+1:3*Hnode+1);
%
%         for pp=1:N
%             yNN(pp)=my_simulate(1,15,1, W,B,xf(pp));
%         end
% %
% disp(['GSA is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(Fbest),')'])
% disp(['CPSOGSA is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(gBestScore), ')'])
% disp(['ACO is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(BestSolACO.Cost),')'])
% disp(['BBO is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(BestSol.Cost),')'])
% disp(['DE is training FNN (Run # = ', num2str(i),' ,MSE = ', num2str(BestSolDE.Cost),')'])

%     end
% end

% % A_Classification_rate=mean(Classification_rate);
%     Average= mean(BestSolutions);
%     StandDP=std(BestSolutions);
%     Med = median(BestSolutions);
%     [BestValueP I] = min(BestSolutions);
%     [WorstValueP IM]=max(BestSolutions);
% end

% figure
%         set(axes,'FontName','Times New Roman');
%         hold on
%         grid on;
%         xfDenorm = mapminmax('reverse',xf,PS);
%         yfDenorm = mapminmax('reverse',yNN,PS2);
%         test_error(1,i)=test_error(1,i)+sum(abs( yfDenorm- yf1 ));
%         A_Test_Error=mean(test_error);
%         plot(xf1,yf1,'DisplayName','Real curve','Color','b');
%         plot(xfDenorm,yfDenorm,'DisplayName','Approximated curve','Marker','.','LineStyle','-','Color','r');
%         %eqtext = '$$sin(2x)e^{-x}$$';
%
%         name='GSA'
%
%         title([['\fontsize{12}\it ', name]],'FontName','Times New Roman');
%         xlabel('\fontsize{12}\it X');
%         ylabel('\fontsize{12}\it Y');
%         legend('toggle');
%         set(legend,'FontAngle','italic','FontName','Times New Roman')

disp(['Best=',num2str( BestValueP)])
disp(['Worst=',num2str(WorstValueP)])
disp(['Average=',num2str( Average)])
disp(['Standard_Deviation=',num2str( StandDP)])
disp(['Median=',num2str(Med)])
% % disp(['Mean_Test_Error = ' , num2str(A_Test_Error)])

figure
semilogy(1:Max_iteration,BestChart,'DisplayName','GSA','Color','g','Marker','o','LineStyle','-','LineWidth',2,...
'MarkerEdgeColor','g','MarkerFaceColor',[.49 1 .63],'MarkerSize',5);
hold on

semilogy(1:Max_iteration,GlobalBestCost,'DisplayName','CPSOGSA', 'Color', 'r','Marker','diamond','LineStyle','-','LineWidth',2,...
'MarkerEdgeColor','r','MarkerFaceColor',[.49 1 .63],'MarkerSize',5);
semilogy(1:Max_iteration,BestCostACO,'DisplayName','ACO','Color','c','Marker','square','LineStyle','-','LineWidth',2,...
'MarkerEdgeColor','c','MarkerFaceColor',[.49 1 .63],'MarkerSize',5);
semilogy(1:Max_iteration,BestCost,'DisplayName','BBO','Color','b','Marker','*','LineStyle','-','LineWidth',2,...
'MarkerEdgeColor','b','MarkerFaceColor',[.49 1 .63],'MarkerSize',5);
semilogy(1:Max_iteration,BestCostDE,'DisplayName','DE','Color','y','Marker','+','LineStyle','-','LineWidth',2,...
'MarkerEdgeColor','y','MarkerFaceColor',[.49 1 .63],'MarkerSize',5);

title ('\fontsize{12}\bf XOR Dataset');
% title ('\fontsize{12}\bf Baloon Dataset');
% title ('\fontsize{12}\bf Iris Dataset');
% title ('\fontsize{12}\bf Cancer Dataset');
% title ('\fontsize{12}\bf Heart Dataset');
% title ('\fontsize{12}\bf Sigmoid Dataset');
% title ('\fontsize{12}\bf Cosine Dataset');
% title ('\fontsize{12}\bf Sine Dataset');

xlabel('\fontsize{12}\bf Iteration');
ylabel('\fontsize{12}\bf log(MSE)');

legend('\fontsize{10}\bf GSA','\fontsize{10}\bf CPSOGSA','\fontsize{10}\bf ACO','\fontsize{10}\bf BBO','\fontsize{10}\bf DE',1);

axis tight
box on



## 4. References

Rather, S.A. and Bala, P.S. (2020), "A hybrid constriction coefficient-based particle swarm optimization and gravitational search algorithm for training multi-layer perceptron", International Journal of Intelligent Computing and Cybernetics, Vol. 13 No. 2, pp. 129-165. https://doi.org/10.1108/IJICC-09-2019-0105

Posted on Sun, 05 Sep 2021 03:34:30 -0400 by freakuency