Understanding how the writing systems are related may be useful for gauging how much overlap there is.
Basically, the Chinese writing system was loaned into Japanese, with various rules and conventions at various times.
In brief, some characters were just "lifted" to have the same meaning in both languages, although the languages are completely unrelated (other than via cultural osmosis) but completely different pronunciations. These symbols are called kun. Others were applied differently, usually by loaning both a word's writing symbol and its pronunciation; but because the languages are so different, the Japanese pronunciation ended up sounding different (and often, Chinese has changed markedly since those times, so that the modern pronunciation is also quite different now). This is called on. There are multiple different on strata, sometimes for the same symbol.
An illustrative complexity in kanji are the radicals. The radicals in Chinese are sometimes chosen simply to suggest how a word is pronounced. But in Japanese, where the word often sounds completely different, these radicals have no mnemonic value; they are just another accidental quirk you have to learn (somewhat similarly to how foreign or ghost are spelled weirdly if you think about it; but this is just a fact of life which you internalized when you learned to write them).
I don't speak either language, but here is a simple example adapted from a site about Chinese radicals:
包 bāo, which means package, is made of 2 radicals:
- 勹 bāo (wrap), is the phonetics radical
- 巳 sì (year of the snake)
But the Japanese word is tsutsumi where obviously the phonetics radical offers no clue whatsoever.
As a rough parable, you don't speak Latin or Greek just because you speak English, or vice versa. English has loaned heavily from classical languages (and French, and Old Norse) so you will recognize many words and have some idea of what they mean – civis or polity are reasonably transparent if you want to start to learn classical languages, but many other words have unrecognizable forms or distinctly different meanings in English than in the languages they originated from. (But all of these are Indo-European languages, and so they share many lexical and structural traits. Japanese and Chinese are much more different from each other, more like Hungarian or Basque compared to English ... or, indeed, Chinese or Japanese.)
Tangentially, I happen to know Finnish, and I find it amusing to trace words which were loaned into a form which is hard to recognize. Some examples from this realm might help you understand how Chinese words might have been similarly adapted to Japanese. I'll bring up kauppa for "shop" (where you can perhaps recognize German kaufen or Icelandic kaupa which in modern Swedish is "köpa" and pronounced roughly "sher-PA") and ranta for "shore" or "beach" which is strand in German and Swedish. The adaptations to make the words suitable for a Finnish mouth are sometimes rather crude. The Finnish syllable system is more complex than the Japanese one, but similarly restricted in what combinations of sounds it permits.
Returning to your question; compared to someone who is unfamiliar with the writing system, you do have a head start; the writing system is a major stumbling block for many people who want to learn Chinese.
(This answer was inspired by the excellent exposition in Geoffrey Sampson's book Writing Systems: A Linguistic Introduction which devotes an entire chapter to the Japanese writing system and its evolution and quirks. I'm flabbergasted that literacy in Japanese is common at all, let alone higher than in many countries.)