I addressed this issue in another question, where I wrote, among other things :
In his video on learning foreign alphabets, Olly Richards mentions some (anonymous) learners of Japanese, Chinese and Cantonese who advise not to learn kanji or hanzi at the beginning because it is a lot of work and takes time away from learning to speak. (Also, it is neither useful nor motivating to learn characters for words that you don't know how to use correctly in a communicative context.)
On Language Log, Victor Mair, professor of Chinese Language and Literature at the University of Pennsylvania, wrote in Februar 2014 (emphasis added):
If I were the czar or god of Chinese and Japanese language pedagogy, I would not teach students a single Chinese character until they were relatively fluent — about two years. I've always said that we should learn languages the way babies do; they learn to speak long before they learn to write.
In fact, this approach was used at a few American universities (Penn State University and Ohio State University). Victor Mair also cites a study by Jerome L. Packard (1990):
Jerry found that the time lag of delayed character introduction improved students' ability to discriminate Chinese sounds, and improved their fluency.
Two other interesting posts on Language Log are The future of Chinese language learning is now (5 April 2014) Learning to read and write Chinese (11 July 2016).
Even though the above comments are mainly about Chinese, I think they also apply to Japanese. So if you want to learn to speak before learning to write, you are in good company. However, you may have reasons to learn speaking and writing at the same time, for example:
- Your job — or other goals that motivate you to learn Japanese (e.g. if you want to read mangas) — require that you learn to read and write Japanese.
- The school where you are learning Japanese teaches reading and writing along with oral skills from day one.
- You want to learn to read older forms of Japanese (e.g. Early Modern Japanese, Middle Japanese).
If you are in control of your learning, I think the first part of my answer gives you good arguments to learn oral skills first.
Update: The textbook series Japanese: The Spoken Language, written by Eleanor Harz Jorden and published by Yale University Press in the late 1980s, focused exclusively on the spoken language.