Introduction to: NWC603COM – Using AI in Computer Games Assignment 2

Introduction to: NWC603COM – Using AI in Computer Games Assignment 2


Artificial intelligence (AI) in the gaming world today has become the sole reason games become either a massive success, or flop, and become a failed enterprise. AI can make or break a game, and the methods and techniques used by developers to give us the illusion that AI is intelligent is what we will be investigating in this blog. (Introduction to: NWC603COM – Using AI in Computer Games Assignment 1)

In this assignment I will be following up from assignment 1, now I have researched the AI techniques used in computer games, I will be first thinking of a simple game idea and then I will talk about the AI algorithms\techniques applied and how they will used in-game. For the techniques that I wont be applying to my game idea, I will be talking about how they are used in other games we play today. I have decided to fulfil this task ahead by creating a blog, I feel I can give you the reader, a much more personal opinion of my views of AI in Computer Games.


AI Techniques I will be reviewing:

1.1 My Game Idea
1.2 Algorithm & Hacks
1.3 Tactics & Strategy's
1.4 A*
1.5 State Machine
1.6 Patrol

2.1 Elements not used in my game
2.2 Boids Flocking
2.3 Fuzzy Logic & Decision Tree's
2.4 Q-Learning & Artificial Neural Networks
2.5 Heuristics

3.1 Conclusion
3.2 References

Introduction to: NWC603COM – Assignment Brief

NWC603COM Assignment Two 

AI and Games

Learning Outcomes to be assessed in this assignment

Evaluate suitable core AI elements and their suitability to different game contexts and then generate ideas and concepts to use AI elements in a computer game.. 

Portfolio - Documentation and oral examination (viva)

This portfolio represents 20% of the module mark.

Task

Blog your game ideas and in particular how AI can be used within it. Describe the various elements you are doing and how they apply to this concept you are proposing.

So in basic terms talk about all the elements and how they could be used in a game and then for the main element focus on it in more detail pls.

This document / blog should also contain information on what type of game you are planning for the assignment and what techniques you deem suitable.

Viva

In a short viva you will be asked about your game concept and the various AI elements. Number of questions about the previous weeks’ topics, aimed at testing your understanding as well as to explain your game design ideas, described on your document / blog.

3.1 Conclusion

Conclusion

So in my second assignment, I was first tasked to come up with a game idea, one that include one or more algorithmic elements of gaming AI, I then had to create a blog which covered the algorithmic elements to be used in my game. I was then tasked to research the remaining elements that I don't plan on using in my game but to discuss where and how they are used in today's games. The AI element in section 1 which covers the methods used in my game idea are: Algorithm & Hacks, Tactics & Strategy's, A*, State Machine and finally Patrol. The elements discussed not used in my game where that of: Boids Flocking, Fuzzy Logic & Decision Tree's, Q-Learning & Artificial Neural Networks and Heuristics. My intent was to not only find out in more depth what many elements I will be using in my game idea but understand how they work with the great games I play everyday.

I'd like to just refer back to my game idea post (1.1 My Game Idea)

My idea is to have a very basic layout of lets say two rooms, my player character will start in room A and I will have a zombie AI character in room B. The goal is so that the zombie character will first walk around room B on a patrol walking between two points in the room and as the player character then enters the room B, the zombie will then trigger, chase down and kill the player character.

From the above plan of my game idea, I would like to walk through very quickly where I have succeeded in my task of applying these AI elements to my zombies.

Algorithm & Hacks - So the basic rules that have been tasked to the zombies is that of algorithm, from patrolling between two points to the attack on the player character. the hack comes in when the player can predictability of the AI, also the very similar code applied to both zombies (lazy code).
Tactics & Strategy's - When the AI's move in to attach the player, they will work together to surround and kill the player character.
A* - This element will come into play when the zombies leave their position to chase down the player, in the way will be a rock so the AI will need to determine the best route for moving around the rock and getting to the player character.
State Machine - Used when the character player enters the room and the zombie AI will go from a wander state (patrolling between two points) to a attack state and chasing down the player.
Patrol - Will be used right in the beginning when the zombies are patrolling between point 1 and point 2 in room B.

I can honestly say that from this blog, I have learnt that even in the simplest of games, there are many AI elements that take place. I've been again able to research these aspects of AI and the roles they partake in the games that are enjoyed by millions. To finish I would like to say that all elements begin with an algorithm, the basic rule setting technique is applied in all AI's in games and without that function, we wont have developed the amazing AI elements we have today to make games so realistic.

3.2 References

References

(1) Company Facts, Unity - Game Engine, 2017, https://unity3d.com/public-relations

(2) W Gallear, AI and Games, Waynes HE Blog, 2017, https://hnchnddegree.blogspot.co.uk/2017/10/ai-and-games.html7

(3) A Champandard, RTS to FPS: Using Strategy AI in Action Games, AI Game Dev, Oct 5th 2007, http://aigamedev.com/open/article/rts-fps/

(4) A Champandard, RTS to FPS: Using Strategy AI in Action Games, AI Game Dev, Oct 5th 2007, http://aigamedev.com/open/article/rts-fps/

(5) J Grzyb, Artificial Intelligence in Games, Code Project, Aug 28th 2013, https://www.codeproject.com/Articles/14840/Artificial-Intelligence-in-Games

(6) J Weibull, Visualising game theory with boids flocking, YouTube, June 1st 2015,
https://youtu.be/Tz3CgSwUREQ

(7) Unity Documentaiton, Making an Agent Patrol Between a Set of Points, Jan 2017
https://docs.unity3d.com/Manual/nav-AgentPatrol.html

(8) C Jay, Drivatars explained, YouTube, Nov 26th 2013, https://www.youtube.com/watch?v=MpMi93bBZOE

1.1 My Game Idea

My Game Idea

The software I will be using to create my game will be Unity - Game Engine, "Unity Technologies offers a platform for creating beautiful and engaging 2D, 3D, VR, and AR games and apps. A powerful graphics engine and full-featured editor enable you to realize your creative vision fast, and deliver your content to virtually any media or device. You can easily connect to your audiences on PCs, consoles, the web, mobile devices, home entertainment systems, embedded systems, or head-mounted displays."(1) Unity is a massively used product in industry around the world, over 34% of the worlds top games are made using the Unity - Game Engine. PlayStation, Apple IOS, Android, Microsoft are just some of the many platforms Unity supports, because of its world known name and mass use in the industry today, I have chosen to use Unity as my platform for creating my game, I feel it is important to learn how Unity works and gain experience using one of the worlds most used engines.

So my game... My idea is to have a very basic layout of lets say two rooms, my player character will start in room A and I will have a zombie AI character in room B. The goal is so that the zombie character will first walk around room B on a patrol walking between two points in the room and as the player character then enters the room B, the zombie will then trigger, chase down and kill the player character. Below is my layout of the game and how I want it to look:

So as you can see from my floor plan that the player character will start in room A and head towards room B, room B is being guarded by two zombie AI's which is currently patrolling between two points. When the player character enters room B the AI's vision path will detect the player character, chase him down and kill him. 

The elements of gaming AI I will be using to make this game with be that of algorithm (Using AI in Computer Games - Assignment 1, 1.1 Algorithms), because I am setting the AI character's basic rules to follow. By telling them to walk between two points and then hunt down the player character and kill him when noticed in game triggering an instance, simple commands telling the AI what to do. I will also be linking this to a hack algorithm (Using AI in Computer Games - Assignment 1, 1.3 Hacks) in gaming AI, in this instance the hack will be applied to both zombie AI's because the code will be the same, the code is so basic that the player character will know what the zombies will do even before it has happened.

Another element applied will be that of tactics and strategy, (Using AI in Computer Games - Assignment 1, 5.1 Tactics & Strategy's) now there will not be much but the zombie AI's will have a strategy which will be to surround and attack the player character. 

State Machine will be used in my game to refer to the state of the zombie AI's (Using AI in Computer Games - Assignment 1, 4.1 State Machines) this could be either evade, attack, wander an find aid, the two I will be covering will be wander and attack, two very important elements which will be linked in my game.

The final element is a pathfinding concept, the A* heuristic cost function algorithm will be used in the zombie AI. The reason the A* will be used is because Dijkstra can find the most ideal paths from one node to all other nodes on the graph, but A* finds only one good path between two nodes. And that is what I'm looking for, there will only be one objective for the zombie which will be to walk between two points and the attack the player character.

So there we go, I now have my game idea, what engine I will be working in, the layout for the game and finally the algorithm elements that I will be using to make the game work. In the next 1.2, 1.3 and 1.4 I will be discussing the three elements used in greater detail. In section 2 I will be covering the other elements used in AI in gaming and see how I could or could have applied them into my game along with how they are used in the games we play today.

1.2 Algorithm and Hack

Algorithm

So where will the algorithm element fit into my game? Well I now have my game planned out as you can see in 1.1 Game Idea, the main element of the zombie AI will be that of algorithm. The first action of the zombies will be to walk between two points in room B, like on a patrol. The algorithm aspect will be basic set of rules that I am giving the zombie AI characters, you can see my post on algorithms from my previous post (Introduction to: NWC603COM – Using AI in Computer Games Assignment 1 - 1.1 Algorithm).

("Simple set of rules of the form of an algorithm to simulate real world behaviour. If we look at games form the past such as pac-man. If we look at the behaviour of the ghosts. They move around the mase is a seemingly random motion then once the player gets close they simply chase the player") (2)

So lets compare with from what Gallear is quoting to my game, well he says that algorithms are used to simulate real world behaviour, I would agree to him to an extent, yes the zombies will be moving like humans, putting one leg forward and then the other but being zombies... they don't exist so it cant be exactly like real world. Gallear refers to the game pacman and how the ghost AI responds to the player character, on my game I want to emulate the same intention but whereas in pacman, when the player character gets to close then the AI will attack, in my game I plan to have the AI chase down the player character immediately when the player character enters room B. So lets look into the role of the zombie AI's:

Zombie A - Will be set to patrol between point 1 to point 2 in room B, when the character player enters room B the zombie AI is then instructed to chase down and attack the player character.
Zombie B - will have the same commands as zombie A but will be tasked to patrol from point 2 to point 1.

So these rules I will be putting in place will be simple yet effective in my game, the algorithms will simulate the real word if zombies were to take over. It has the excitement factor, scariness and killing side of a game which fits the target audience of a lot of current day gamers.


Hack

The hack in this game will apply to the two zombies that there actions will be so predictable, the player character will know what the zombie plans to do before it has even happened. so to emphasise into a little more detail, the code applied to the zombie 1 will be the same applied to zombie 2, otherwise known as lazy code. This is not necessarily a hack but does make a difference in the end user experience, by just doing a little extra and applying a different objective to zombie 2 can make the difference in a game. Yet again hacks can also be useful, so in my game we have two zombies and you need to remember that zombies don't think, the only thing they know is how to chase down and eat the enemy (player character). So because of this if you apply a hack or same code (lazy code) to all of the zombies, if they are predictable and you know what the zombie AI plans to do before it happens then in this case good, zombies are predictable, we know what they are going to do before it happens so in this situation, a hack is a positive implement in this game.

In my first assignment I cover hacks in gaming AI (Introduction to: NWC603COM – Using AI in Computer Games Assignment 1 - 1.3 Hack). I cover where you can find hacks in a game, good and bad but I really need to cover the hacks involved in my game. So I will be apply very similar code to both of my zombie AI's, the only difference is when the zombies are patrolling between the two points in room B. Now as a end user (player) you expect the zombies to just come chasing after you, and they'll be rights, its a simple hack yet a positive hack because it emulates a zombie down to a tea because there predicable and brings a real world experience - well if zombies were to take over the world.

1.3 Tactics & Strategy's

Tactics & Strategy's

The element of using tactics and strategy's in gaming AI is one of importance and vision in the role of emerging the player into a game, what I mean by this is by using the element of tactics and stratigy's in game, it brings a game realism to life by imitating real life situations. I have talked about tactics and strategy's in a previous post in the my blog, (Introduction to: NWC603COM – Using AI in Computer Games Assignment 1 - 5.1 Tactics & Strategy's) in this post I talk about how tactics and strategy's are used in today's games and the role they have, in this post I will be looking at how tactics and strategy's will apply to my game idea.

"Because of this design choice, level designers effectively fill the role of the high-level AI in real-time strategy games. Level designers:

• Are responsible for placing enemies strategically.
• Must balance the difficulty settings for each level.
• Implement scripts to handle special situations." (3)

Now the initial idea of strategy in AI is setting the AI a plan of action for or in game, now Champandard makes the first point in that strategy for AI is used for placing the enemy AI in a strategic position, now my game only has two AI zombie but they will be placed in a position where they can one, reach each point and then as soon as the player character enters room B, they can attack without any obstacle in its path. Now my game is not in the greatest detail when it comes to setting and changing the difficulty of the AI, the example used by Champandard is that of a game that has levels meaning the higher level you go the harder the AI is to kill.When the environment changes to that which  requires special situations, this could be when a member of the AI team gets kill in action, for example the leader of the team could get taken out so in that case another member of the team would need to step up and become the leader of the group. Again in my game this type of strategy is not used because the their are only two AI zombies working together. 

"In games like Halo and Killzone, paths are computed with a custom heuristic. This means that combat factors are included when selecting the best path:

• Is the actor moving through the cone of fire of a teammate?
• Is the actor exposed to the player’s weapons?"(4)

Now I have kept with Champandard because he gives a great view on tactics in AI gaming, the element of tactics will be applied in my game because the goal of the Zombie AI is to attack and kill the player character, the two zombies will work together and bet set a tactic to surround the player character to each side which disables him from escaping the scene. The quote also does say that tactics link in with heuristics because the AI used the best path, a common trial and error process to gain the best result and decides which direction the AI will head next. You could say that based on the current tactic of the AI was to get to a location without being detected by the player character but in order to do this the AI would need to use heuristics to find out which path would be the best and would work, other pathfinding options would be to find the least expensive route but this might not be the right move as it could lead in the AI being detected.

Its important to factor into my game that the element of tactics and strategy's wont play a massive role in the final outcome of my game but having the zombies use a strategy to surround the player character instead of both using the same path which could allow the player character to escape the situation I feel makes the scene that little more wild for the user playing and adds difficulty to the game without apply a higher damage rate required to kill one of the zombies.

1.4 A*

A* Algorithm

The A* algorithm (Using AI in Computer Games - Assignment 1, 3.1 Pathfinding Concepts - A* Algorithm) the process of plotting an efficiently directed path between multiple points, So where will this be used in my game idea you ask...

Now the A* algorithm will come into play when the player character enters room B, the zombie AI's will then leave their patrol position and begin to chase down the player character. As you see in the image above that in the path of the AI's is a rock, the zombies will need to use pathfinding - a* to move around the rock and get to the player character. The AI's will use the A* algorithm to plot a directed efficient path to the player character, this will add more realism to the game and will evade the attempt of the zombie AI running into the rock.

"A-Star algorithm, used in performing fast searches for the optimal path connecting two points on the map (graph) of a game."

"That time witnessed the birth of such memorable games as the immortal River-Raid, Donkey-Kong, Boulder-Dash, and many other objects of fascination for users of eight-bit machines, back in the 1970s."(5)

Grzyb makes the point that the A* algorithm has been used in the games we play today for more than 40 years, they are the soul purpose which gives the AI the ability to avoid objects and get from point to point on the map. Its a simple applied algorithm which can enhance a games realism just by allowing the AI to move how a human should.

1.5 State Machine

State Machine

The AI element of the state machine (Introduction to: NWC603COM – Using AI in Computer Games Assignment 1 - 4.1 State Machine) is one that is applied to an AI which during game play will have to to have at least two of the four states evade, attack, find aid and wander. Now in my game I will be applying two of these states to my zombie AI's so that's the two I am going to focus on.

Wander - From the wander state, the AI can then move into attack or find aid. When the game begins, the two zombie AI's will be patrolling from two points in room B, during this time they will be in a wander state, it is a relaxed no threat, no danger state but the AI will always stay alert.

Attack - The attack state will be used in my game when the player character enters room B which will trigger the zombies to move from a wander state into an attack state and move it to kill the player character, these two states combined can transform a stable patrolling AI into a killing machine as soon as the player character is near, you can see this from the image above.

Evade - One of the two other status's of the state machine s the evade state, this is when the AI has been in battle with the player character and to escape from the situation, the AI will move into an evade state. This really means to run away and find a safe location. Now this state is not used in my game.

Find aid - Find aid is the final state and again, not used in my game. This state is used when the AI has taken damage from the player character, to get into a find aid state, first the AI will have had to be in an evade state (run away\hide) from here the AI can be safe to increase the AI's health. Once the AI has restored its health it will then revoke back into a wander state and begin the process again.

Games today where this AI technique is used would be Call of Duty, Battlefield and most shooter games. So once I apply the attack and wander sections of the state machine to my zombie AI's, they will have algorithm to chase down and kill my player.

1.6 Patrol

Patrol

The AI element patrol is one that is used in my game from the get go, the zombie AI's will patrol between two points in room B and by using this algorithm technique it makes the game more realistic instead of two zombies just standing there waiting for the player character to enter the room.

So you can see that the element of patrol has been added here as the zombie AI have been tasked to walk between point 1 and point 2, during this the AI will stay on alert awaiting for the player character to enter the room and from there they will go from a wander state to an attack state (1.5 State Machine).

"The ideal sequence of patrol points will depend on the way you want the NPCs to behave. For example, a robot would probably just visit the points in a methodical order while a human guard might try to catch the player out by using a more random pattern. The simple behaviour of the robot can be implemented using the code shown below."(7)

This quote it from the documentation manual so its hard to argue, but the reason I have added this quote is because they state that a robot will just go from point to point as directed but another more intelligent AI would go off path to get to the same point but to take another route either to avoid another AI, player character or danger factor. It shows us that there is more to this element that just a basic set of instruction telling an agent to go from point A to point B.

Games where you see the patrol element added to AI characters would be Battlefield (soldiers marching between two or more points guarding its territory) or maybe in the game Total War (a unit of soldiers can be given instructions to march from point A to point B to move into position.

So you can see that adding the patrol element can bring a great simulation of real life by giving the soldiers a purpose to be there, there is no need to stand about when they can be made to walk about a look busy. Another small tactic used to emerge for the best player experience.

2.1 Elements not used in my game

Elements not used in my game

As my game is only a basic zombie AI chase down the player character and kill the player character game, not all gaming AI elements are used. To name a few: 2.2 Boids Flocking, 2.3 Fuzzy Logic & Decision Tree's, 2.4 Q-Learning & ANN and 2.5 Heuristics.

In this next area of my blog, I will be talking above the elements and if I were to apply them into my game, where and how could I implement them into my game idea. I will also be talking about what games these AI elements are implemented in and how they are used to make games better.

Its important to cover the methods not included in my game idea, it will allow me to compare and see where I can improve my game idea. Not only that, it will allow me to research where theses elements are used in the games we play and enjoy today.

2.2 Boids Flocking

Boids Flocking

The AI element of Boids Flocking is one this is not used in my game idea, although it is a technique used in many of the games we play today (Using AI in Computer Games - Assignment 1, 4.3 Boids Flocking). Its the element used in games to emerge the player and to make him\her feel like they are actually in the game, the method itself are lots of agents schooling together, making similar movement to emulate a flock of birds or a school fish.

So where have we seen boids flocking and how is it used in the games we play today? We tend to find this technique used in real world games such as Grand Theft Auto, Far Cry and Gran Tourismo.

 (6)

From the attach video you see above, you can see boids flocking used in a perfect setting, now the idea is that you are the player character in a real world situation. The idea is for you to be the 1st person player and you are on a mission, the idea is not to focus on the birds flocking above. But the way boids work is that because your focusing on your task ahead, a random flock of birds flying across the top of the screen is a tactic to emerge you into the game itself, it brings the real world element to life and draws you deeper and deeper into the game.

2.3 Fuzzy Logic & Decision Tree's

Fuzzy Logic & Decision Tree's

Now the use of the fuzzy logic algorithm as been explained in my previous blog (Using AI in Computer Games - Assignment 1, 4.2 Fuzzy Logic) So I will include my first section from that post "where the state membership has a float range and not made up of 0 and 1's, what I mean by this is the state is a number between the 0 and 1, so for example 0.5. You're probably thinking, what does this mean? Well to make it simpler, in a shooting game the AI as well as the player character will have a health bar, below is an example of Fuzzy logic...

"

Now the algorithm fuzzy logic is not used in my game, although the act of attacking and then death is part of my game, I do not intend to put in the technique of fuzzy logic. If I was to apply fuzzy logic to my game then the zombie AI is where I would apply it to, why? Well because as you can see from my image above that fuzzy logic is used to portray that of a health bar, by assigning this algorithm to the AI it will give the zombies a life span. In my game the player character is helpless and cannot damage the zombie AI so applying fuzzy logic to the zombies would be pointless, but a technique such as this is used in almost if not all shooter games today.

Now I just mentioned fuzzy logic is used in most shooter games, but also used in other genre's too.

Now you are probably wondering why I have uploaded a screenshot from the popular game franchise FIFA, well in the bottom left corner you can see a small green bar under the players (AI's) name. This is a stamina bar which is a player rating used to measure how far the AI can run before it gets tired, when the game starts the green stamina bar will have a rating of 100, as the game goes on the player with become more tired resulting in the bar to reduce. This is a perfect example of a fuzzy logic health bar in action, this will deteriorate quicker the lower the stamina rating or in a shooter, the higher the damage rating.




Decision tree's on the other hand (Using AI in Computer Games - Assignment 1, 4.4 Decision Tree) as mentioned in my previous blog are basically yes or no questions, it's the action of questions that an IA will ask its self depending on its surroundings, current state and health stature... A typical decision tree line of questioning in a shooter game such as Call of Duty for an AI would be:                   
Have I taken any damage? -> Yes -> find a place to hide until healthy -> Do I now have full health?   -> No -> continue hiding until healthy -> Do I now have full health?   -> Yes -> Return to the battle scene.

So where could this fit into my game if I wish to apply the decision tree algorithm? Well as the player character enters the room, the AI state of attack could go like this:
Has the player character entered room B -> No -> continue on patrol between the two points in the room.
or
Has the player character entered room B -> Yes -> Attack player character.

So has a decision tree been used in today's games? Of course, it's used all a majority of games including AI today. It gives the AI its own decision path, own brain so to say, well it doesn't give them the option to make their own path and options but from the options its given and based on the questions asked the AI can make its next move. The AI characters in Call of Duty would use the decision tree when under attack from the player character or even other AI:
Have I taken any damage? -> Yes -> find a place to hide until healthy -> Do I now have full health?   -> No -> continue hiding until healthy -> Do I now have full health?   -> Yes -> Return to the battle scene.

Decision tree's in games are used to give the AI that new leaf on life, when all is at its darkest, the AI can find recovery and return to action. So in away the decision tree will be used in my game as long as it is kept very basic, it will be a simple one question for the AI from the example above and then the game will start over.

2.4 Q-Learning & Artificial Neural Networks

Q-Learning & Artificial Neural Networks

An Artificial Neural Network in gaming AI (Using AI in Computer Games - Assignment 1, 5.2 Learning Technologies - Artificial Neural Networks) is a piece of software which is used to make the AI in games more intelligent, it works with the heuristic method meaning the AI learns from its previous accounts. But unlike heuristics the ANN gains more information which means it can utilise the data it has gathered from previous encounters, it works more like a brain and some go on to say it can begin to out think the human mind.

So where could I apply Artificial Neural Networks to the AI in my game idea? Well in my game I have one player character and two zombie AI's, now when the player character enters room B and gets attacked and killed by the zombie AI's. If I was to allow it so the player can kill the zombies, then the zombie AI would re-spawn in the same location as before but with ANN applied to the AI, the AI will have learned and will remember where the player character is hiding and wont make the same mistake again by running into your path. This would be enough evidence to show that the AI has learnt where the player was where he killed the AI which results in the AI spawning in either a safer location or in a better location in which to attack the player character.

Where are Artificial Neural Networks used in today's games? Well actually ANN's have been applied in games to AI's for a long time now, even games as old as Red Fraction used this technique to make the AI more intelligent and harder for the player character. Newer games such as Battlefield 4 have AI which learns from the way you act as a player, it can learn to predict what you plan to do and then learn to out think you.



Now Q-Learning in gaming AI (Using AI in Computer Games - Assignment 1, 5.3 Learning Technologies - Q-Learning ) is a learning algorithm and the way this works is that an AI is rewarded or punished based on its previous actions. Sort of like a dog, if a dog behaves well, that dog will get a treat, but if the dog pee's in the house, that dog will be in trouble. Q-Learning is so linked to human life and ANN's that it combines the human way of life and apply's that to a gaming AI but I have already explained all this in my previous blog, so what we really want to know is if Q-Learning can be applied into my game.

Well for Q-Learning to work, i'll need to combine the two processes of trial and error and the process breaking a problem down into smaller problems. But because my game idea is very basic, to apply Q-Learning would take much more AI involvement and a bigger plot to be set. It links in with Artificial Neural Networks because of its heuristic approach, if I could make the zombie AI's attack each other but because they are not supposed to, this would result in a bad god situation and the zombie AI being punished. Now the AI could lose health, points or even die of its own actions.

(8)

Q-learning is used in more modern games we play today because the advancements in AI is much more in depth then it ever was, a prime example would be Forza on Xbox One, you can create a drivatar which is an AI which learns how you drive in the game by imitating and repeating your driving style. In the video link above, you can see how drivatar is used to improve the players driving skill, which shows advanced AI now just going against the player but assisting the player.

2.5 Heuristics

Heuristics

Heuristics is the basic term of the trail and error process for AI in gaming, the AI characters of a game would act on its previous experience of trail and error (Using AI in Computer Games - Assignment 1, 1.2 Heuristics) A simple element which has been used in AI gaming for years.

Now from the previous posts in this blog, I have covered elements which have heuristics built into them so without heuristics, they would not work. I am referring to Artificial Neural Networks (2.4 Q-Learning & Artificial Neural Networks) Now from my initial game idea post (1.1 My Game Idea) you can tell that there are no heuristics involved, there is no trail and error for the AI to learn from. So where would it fit into my game idea is I was to put in heuristics? Well we can come to that in a short time, first I want to talk about heuristics in today's games...

So heuristics used in a game to would be a AI soldier in a game like Battlefield, if the AI is in an attack state then the AI will charge at the playing character but once the AI was under attack from the player character, then the AI would look for cover, somewhere to hide. If the heuristic method is that of a prediction, then we or even the AI would not know what to do when it was under attack. 

Now looking a heuristics in my game, I would do a very similar act as the AI in battlefield. Once the zombies go into an attack state, they will charge down the player character. Now the adjustment I world need to add in would be for the player character to attack back at the zombie AI's, then the zombies would revert to a evade state in which then would find somewhere to hide from the player. Using the same prediction method, the AI would not know what to do when it was under attack. To cut a long story short, the trail and error method would need to be implemented into my game for the heuristic method to be considered.