It is a well known matter that one of the most important physicist ability relies on her techniques to solve differential equations. As Dirac once said, best physicists are those that solve such equations without doing this explicitely. This gift is required also when one is able to formulate new laws of physics. So, what are today the most used techniques to accomplish this task? One could respond paraphrasing a known ad by a coffee brand: Perturbation theory, what else? There has been some attempts to propose non-perturbative methods but today the only meaningful one that can be considered as a substantial paradigm is the renormalization group. But there is a problem here. We are only able to evaluate the non-perturbative equations of the renormalization group using perturbation theory. This is somewhat paradoxical. Landau took so seriously this fact that he pretended the existence of a pole due to the beta function of QED. But this is blatantly wrong as one cannot use a first order computation to claim a full solution property. This story of the Landau pole is still alive with us and the reason is that small perturbation theory is such part of our way of thinking to believe that results obtained with this method should be generally true.
In order to have an example of such widespread prejudice one should look at renormalization. Why should one believe that theory at very large coupling produces infinities that are only the product of the computation method? What I expect is that the renormalization constants must be all finite numbers if only I would have the exact solution of my field theory. Infinities are due to the method, i.e. small perturbation theory. This caused some confusion between the physical limits of a theory and the method. But the question is does another method exists? My conviction is that perturbation theory is just like Janus and has two faces but, curiously enough, strong perturbation theory recovers semiclassical limit and in this case one should have negligible quantum fluctuations and no renormalization seems to be needed (e.g. see here or Barry Simon’s book here). But having the other face of Janus gives a further strength to renormalization group making it more effective in finding fixed points.
An example of this way of thinking is seen in the use of functional methods for Yang-Mills theory. It is well known that in quantum field theory one can get a tower set of equations, the Dyson-Schwinger equations, that are inherently non perturbative. How could I truncate this tower set? People use to evaluate a truncation point using Feynman diagrams also for a non perturbative regime! Surely, if you are lucky you can also get the right approximation but if this does not happen? A clear understanding of the right truncation in this case can only be obtained using numerical methods to be compared with lattice results. Currently, in this comparison, Aguilar and Natale hit the goal (see here) while others failed. This paper will be part of the history of our understanding of Yang-Mills theory in the infrared.
My conclusion is that is always better to take an open mind when use perturbation methods and anyhow be sure of the approximations you are doing to be certain you are in the regime you pretend to be.