How exactly does one create a chip specialized for an algorithm like fast fourier transforms? Are there logic and arithmetic gates that are just simply made into the silicon chips themselves? Seems kinda hard to actually make something like this right, zo how does a company go about to actually design one of these for a special purpose, be it this algorithm or other use cases like camera or microphone sensors for wake-up detection like Siri.

Designing such specialized hardware is (in my opinion) much simpler than designing CPUs themselves. If you think about of an algorithm as a set of instructions and control dependencies, you can implement it by using gates and FSM(s). There are actually a number of ways (e.g. high level synthesis (HLS)) that even allow you to write a high-level code implementing the logic for your specialized hardware and generate hardware description based on that.