I finally got around to reading a book I've had in my library for some years now and am sorry that it took me so long to get to it. E.A. Burtt's Metaphysical Foundations of Modern Science is a superb analysis of the sources and content of Isaac Newton's philosophical thought, and thus an incisive critique of the entire enterprise of science and modernity.
The first two thirds of the book explicates the thought of Newton's influential predecessors, and the last third is about the thought of Newton himself. There are many great passages throughout the book, but in this post I will confine myself to excerpting from the parts that most directly deal with Newton as a metaphysician.
[I]n his decisive portrayal of the ultimate postulates of the new science and its successful method as they appeared to him, Newton was constituting himself a philosopher rather than a scientist as we now distinguish them. (33)
[Newton] disliked hypotheses, by which he meant explanatory propositions which were not immediately deduced from phenomena. At the same time, following his illustrious predecessors, he does give or assume definite answers to such fundamental questions as the nature of space, time, and matter; the relations of man with the objects of his knowledge; and it is just such answers that constitute metaphysics.1 (33)
An interesting question remains however to be asked about [Newton's] method. Do not the very initial experiments and observations, as a result of which the mathematical behavior of phenomena is defined, presuppose something which we can only speak of as an hypothesis, to direct those experiments to a successful issue? (223)
This section, in which Burtt expresses the profound and ultimately insurmountable problems with the positivism Newton left us, is worth quoting at length:
 To begin with, there is no escape from metaphysics, that is, from the final implications of any proposition or set of propositions. The only way to avoid becoming a metaphysician is to say nothing. This can be illustrated by analysing any statement you please; suppose we take the central position of positivism itself as an example. This can perhaps be fairly stated in some such form as the following: It is possible to acquire truths about things without presupposing any theory of their ultimate nature; or, more simply, it is possible to have a correct knowledge of the part without knowing the nature of the whole. Let us look at this position closely. That it is in some sense correct would seem to be vouched for by the actual successes of science, particularly mathematical science; we can discover regular relations among certain pieces of matter without knowing anything further about them, The question is not about its truth or falsity, but whether there is metaphysics in it. Well, subject it to a searching analysis, and does it not swarm with metaphysical assumptions? In the first place it bristles with phrases which lack precise definition, such as "ultimate nature", "correct knowledge", "nature of the whole", an assumptions of moment are always lurking in phrases which are thus carelessly used. In the second place, defining these phrases as you will, does not the statement reveal highly interesting and exceedingly important implications about the universe? Taking it in any meaning which would be generally accepted, does it not imply, for example, that the universe is essentially pluralistic (except, of course, for thought and language), that is, that some things happen without any genuine dependence on other happenings; and can therefore be described in universal terms without reference to anything else? Scientific positivists testify in various ways to this pluralistic metaphysic; as when they insist that there are isolable systems in nature, whose behaviour, at least in all prominent respects, can be reduced to law without any fear that the investigation of other happenings will do more than place that knowledge in a larger setting. Doubtless, strictly speaking, we could not say that we knew what would happen to our solar system if the fixed stars were of a sudden to vanish, but we do know that it is possible to reduce the major phenomena of our solar system to mathematical law on principles that do not depend on the presence of fixed stars, and hence with no reason to suppose their disappearance would upset our formulations in the least. Now this is certainly an important presumption about the nature of the universe, suggesting many further considerations. Let us forebear, however, to press our reasoning further at this point; the lesson is that even the attempt to escape metaphysics is no sooner put in the form of a proposition than it is seen to involve highly significant metaphysical postulates.
For this reason there is an exceedingly subtle and insidious danger in positivism. If you cannot avoid metaphysics, what kind of metaphysics are you likely to cherish when you sturdily suppose yourself to be free from the abomination? Of course it goes without saying that in this case your metaphysics will be held uncritically because it is unconscious; moreover, it will be passed on to others far more readily than your other notions inasmuch as it will be propagated by insinuation rather than by direct argument. That a serious student of Newton fails to see his master had a most important metaphysic, is an exceedingly interesting testimony to the prevailing influence, throughout modern thought, of the Newtonian first philosophy.
Now the history of mind reveals pretty clearly that the thinker who decries metaphysics will actually hold metaphysical notions of three main types. For one thing, he will share the ideas of his age on ultimate questions, so far as such ideas do not run counter to his interests or awaken his criticism. No one has yet appeared in human history, not even the most profoundly critical intellect, in whom no important idola theatri can be detected, but the metaphysician will at least be superior to his opponent in this respect, in that he will be constantly on his guard against the surreptitious entrance and unquestioned influence of such notions. In the second place, if he be a man engaged in any important inquiry, he must have a method, and he will be under a strong and constant temptation to make a metaphysics out of his method, that is, to suppose the universe ultimately of such a sort that his method must be appropriate and successful. Some of the consequences of succumbing to such a temptation have been abundantly evident in our discussion of the work of Kepler, Galileo, and Descartes. Finally since human nature demands metaphysics for its full intellectual satisfaction, no great mind can wholly avoid playing with ultimate questions, especially where they are powerfully thrust upon it by considerations arising from its positivistic investigations, or by certain vigorous extra-scientific interests, such as religion. But inasmuch as the positivist mind has failed to school itself in careful metaphysical thinking, its ventures at such points will be apt to appear pitiful, inadequate, or even fantastic. Each of these three types is exemplified in Newton. His general conception of the physical world and of man's relation to it, including the revolutionary doctrine of causality and the Cartesian dualism in its final ambiguous outcome (which were the two central features of the new ontology) with their somewhat less central corollaries about the nature and process of sensation, primary and secondary qualities, the imprisoned seat and petty powers of the human soul, was taken over without examination as an assured result of the victorious movement whose greatest champion he was destined to become. His views on space and time belong in part to the same category, but were in part given a most interesting turn by convictions of the third sort. To the second type belongs his treatment of mass, that is, it gains its metaphysical importance from a tendency to extend the implications of his method. Of the third type, mainly, are his ideas of the nature and function of the ether, and of God's existence and relation to the world uncovered by science. We can hardy do better than allow this analysis of the three types to furnish us with an outline of the succeeding sections.
Newton's genius brought mankind great success, but at a great cost that we his children are still paying. Our liberation from nature has enslaved us to ill-conceived philosophy.
Magnificent, irrefutable achievements gave Newton authority over the modern world, which, feeling itself to have become free of metaphysics through Newton the positivist, has become shackled and controlled by a very definite metaphysics through Newton the metaphysician. (230)
The book might be characterized as a reflection on the unexamined presuppositions of modern science, and it does highlight some of the odd religious presumptions on which Newton based his thought (e.g., absolute space as the "sensorium" of God). If one is to believe Wikipedia, Burtt was one of the writers of the first Humanist Manifesto, but not a signatory. Nevertheless he is a broad enough thinker (in some ways comparable to Whitehead) that his own views never intrude to cloud his judgment; I spent the first half of the book thinking he was a mild ally of scientism, and the second half thinking he was a moderate Aristotelian! That he was a professor at Cornell may not be his only similarity to Carl Sagan—but I write no more on that now.
1. That Newton assumed a metaphysics (or more properly a philosophical physics, a.k.a. a natural philosophy) rather than drawing it from rational reflection on nature is what sets Newton's work as far less general than Aristotle's, and far inferior in actual knowledge of the world. It is the reason that his mechanics required correction in the 20th century (viz., quantum mechanics and relativity)—and even these corrections are incomplete.
E.A. Burtt, Metaphysical Foundations of Modern Science (Garden City, NY: Doubleday and Company, 1954). Same pagination as Dover Books edition.