Unfortunately when it comes to image-heavy algorithms like GAN, they are very difficult to run in reasonable time on CPU. My best advice would be to try finding a GPU to train the network with. If you need to use a CPU, your options include decreasing image size / kernel size. Another option is to attempt to utilize dilated convolutions to improve performance. Hope you are able to get something working!