Electromagnetic (EM) simulation is an important tool for modeling and studying high frequency systems in modern industry. However, the solver part in EM simulations represents a serious bottleneck because its execution time rapidly increases as number of equations-to-solve increases. Although several existing research has proposed parallel hardware solvers, there still exists a room to improve the speed and scalability of these solvers. In this paper, we present a scalable architecture that can efficiently accelerate the solver core of an EM simulator. The architecture is implemented on a physical hardware emulation platform and is compared to the state-of-the-art solvers. Experimental results show that the proposed solver is capable of 522x speed-up over the same pure software implementation on Matlab, 184x speed-up over the best iterative software solver from the ALGLIB C++ library, and 5x speed-up over another emulation-based hardware implementation from the literature, solving 2 million equations.