What exactly do you mean to say when you say failed. What is the score you are aiming, and what score did you achieve? 

There are a couple of things you can do to improve your band in the speaking module. 
Using idioms but relevantly can help improve your score. Similarly, using phrasal verbs and a useful vocabulary helps to fetch you a better band. 

Fluency is another factor. Speak as much English as possible with family friends and groups on social sites. Practice speaking on familiar topics like your hobbies, job, subjects, college, aim, home, family members and so on. Also, there is a wide range of cue cards available online and offline. Practice speaking on them. The more you practice, the more the ideas to express. 

Hiring a professional trainer to achieve your desired band is another option you can look into.

Since you've already attempted IELTS four times, you must have understood that this test needs to be prepared well. IELTS is one such test that requires a lot of hard work and patience. It's okay to fail, but you should be able to convert those failures to success for that you need to structure yourself and should seek for more knowledge and experts' guidance to improve the band score. 

The speaking test is conducted to check your fluency and accuracy in English; whether you can communicate effectively or not! So, work on your fluency and coherence, grammatical range and accuracy, lexical resource, and pronunciation. 

In reality, the only way to improve your fluency in speaking is by speaking a lot; involve yourself in speaking English as entirely as possible.  Some of the simple tips on how to improve English speaking skills are:

First of all, if you have a native speaker to practice with, then nothing like it. Joining a club or a volunteer organization can be a great way to know people informally and to converse effectively too. If that isn't an option, consider choosing a private tutor, there are trainers who can guide you and improve your speaking skills. Don't be shy to express your skills in language and be confident.