Well, if you garner enough attention to get any critics interested in savaging you, you will already be doing well. But critics qua critics are unlikely to savage you for it unless you do it clumsily.
The accusation of cultural appropriation is a relatively new weapon in the culture wars. Indeed, adopting the culture or dress of a group or culture was regarded as a mark of appreciation or solidarity not so long ago. So it is much trickier to say what is going to provoke such a charge, and even if you should take such a charge seriously if it is leveled.
How you use elements of a culture could be as much a factor as what elements you use. Using such elements for purposes of mockery or derision is obviously going to provoke a different reaction from using them respectfully or out of utility. But then again, it depends on who you are mocking. Some groups can be mocked with impunity (Catholics, cowboys, Brits, for example). Some cannot be mocked at all on peril of intense vitriol and social exclusion.
But it is very hard to say exactly what will provoke such a charge. I have not heard of anyone accusing Tony Hillerman of cultural appropriation for writing a mystery series featuring two Navajo policemen. (Not saying they haven't, mind you.) But such accusations can suddenly develop as a tool for promoting the agenda of a particular group.
But few cultures or groups worry about this sort of thing and I cannot imagine the Swedes or the Japanese (who have a very fine traditions of performing western classical music, for instance) ever leveling such an accusation. Most cultures borrow from each other and always have done throughout history. Modern society is largely a product of such borrowings and the innovations that arise from the intersection of ideas and practices. The modern West is inconceivable without the mixing of Indian mathematics, Arabic numerals, Roman law, Greek thought, Jewish spirituality, Christian theology, Norse seafaring, Celtic art, Germanic language, etc. etc. It is not merely the presence of these elements that matters, but how they affect each other and the new things that develop as a result.
Objections come from the margins, from groups struggling with the paradoxes of inclusion and distinctiveness, who may feel that their distinctiveness is being eroded as much by the wider culture's adoption of elements of their culture as by their own people adopting elements of the wider culture. Be aware of these sensitivities, whether you decide that they are well merited or not. No point in picking a fight you don't want by mere inadvertence.