Website is hosted on the server and server are just like a another computer which is located in remotely. Computer need to memory to efficiently run multiple applications at the same time. Server administrators are allocated specific memory size to different applications for PHP.
When WordPress giving and memory exhausted errors, need to be allocate more memory on the server.