The alchemist​ in his laboratory was a popular subject for Dutch painters of the 17th century because it allowed them to show off their skill with light. Mattheus van Helmont’s A Savant in His Cabinet, Surrounded by Chemical and Other Apparatus, Examining a Flask (1670s), one of the splendid plates in Philip Ball’s introduction to alchemy, depicts an alchemist at work, brandishing a beaker of pale blue fluid in one hand, surrounded by the implements of his craft. The room is unkempt: hundreds of glass and earthenware vessels, glinting in the dim window light, are strewn about the floor, along with tatty leather-bound tomes and discarded notepaper. To one side looms the athanor (from the Arabic for oven, at-tannur), a coal-fuelled brick furnace where oils were distilled and metals smelted. Among the flasks, phials and funnels are a number of alembics. The alembic, perhaps the most iconic piece of alchemistic labware, also takes its name from Arabic: al-anbīq refers to the long, sloping neck that protrudes from the instrument’s gourd-like body. It was used for distillation, a stock-in-trade of the alchemists and a process they believed could release the spirit of a substance – the ‘spirit of wine’, for instance, which had medical applications in addition to those for which it’s better known.

A picture of the alchemist’s laboratory from a hundred years earlier – a satirical woodcut by Pieter van der Borcht – depicts the alchemist, his wife and children as apes, surrounded by broken equipment, hopelessly puffing on the coals of their furnaces. In the background, another ape family can be seen at the door of the poorhouse accepting a parcel of food. At a time when alchemy was supposedly in its heyday, the print offers a popular view of the alchemist as a deluded fool, absorbed in his quest for the philosopher’s stone while he and his family sink into poverty. The philosopher’s stone, a mythical substance that could transmute any metal into gold, was the alchemist’s main preoccupation for hundreds of years. In that time, though the stone remained elusive, the alchemical laboratory produced many useful things, including dyes, medicines and porcelain. The recipe for porcelain, which had been imported from China, remained a mystery in Europe until an alchemist at the court of Augustus II, King of Poland, discovered it by accident in 1704 when his clay crucible (containing kaolinite) was transformed by intense heat into white ceramic. In 1669, phosphorus was isolated by a German alchemist, Hennig Brand, who distilled large volumes of urine, believing it to contain the crucial agent of transmutation, until he was left with a soft metallic substance that glowed, ignited in air and gave off a garlicky odour.

Belief in the possibility of chrysopoeia, the artificial production of gold, was based on a theory of the elements first expounded by Empedocles in the fifth century bc and later taken up by Plato and Aristotle. Empedocles proposed that all matter is constituted by the four elements of earth, air, fire and water, and that these elements are not immutable: solid substances could be melted to flow like water, while water could be frozen solid. According to Aristotle, each element possesses two of four distinct qualities – hot, cold, dry and wet (water is cold and wet, fire is hot and dry). One element could be transformed into another by altering its qualities: ‘Water becomes earth (cold and dry) by changing wetness to dryness, and becomes air (hot and wet) by changing cold to hot through heating.’ If an earthy substance could become an airy one, then the transmutation of one metal into another, ostensibly a less drastic change, didn’t seem implausible.

Ball traces the roots of alchemy to the metallurgists of ancient Egypt, who mastered the arts of smelting, bronze-making and iron-working. Papyri containing the trade secrets of fourth-century artisans include instructions for ‘making’ (or faking) silver from copper, and for giving copper ‘the appearance of gold’. Zosimos of Panopolis, a late third-century metalworker from Alexandrian Egypt, also writes of the ‘tingeing’ of metals, which he describes as the transfer of the pneuma (‘spirit’) of one metal to the soma (‘body’) of another. But it was in the work of medieval Arabic alchemists that the art of chrysopoeia acquired a firm theoretical basis, and what we now think of as alchemy took shape.

The eighth-century alchemist Jābir ibn Hayyān (known in English as Geber) claimed that all metals are composed of the same two ingredients in different proportions: sulphur, which is hot and dry, and mercury, which is cold and wet. This was taken as a given by the generations of alchemists that followed, though it’s often unclear whether they meant the elements themselves or something closer to their ‘essences’ (alchemical texts refer to ‘Philosophic Mercury’ and ‘Philosophic Sulphur’). According to Geber, one metal could be transformed into another by altering its proportions of mercury and sulphur to match those of gold. This could be achieved via an elixir, a dry powder that combined mercury and sulphur in the perfect ratio. This elixir came to be known as the philosopher’s stone, or lapis philosophorum, thanks to a Latin text attributed to Geber but probably written by an Italian monk five centuries later. Pseudo-Geber claimed that while certain agents of transmutation could create the appearance of gold, only the philosopher’s stone made the real thing.

The principles behind the philosopher’s stone help to explain some of the more enigmatic alchemical art. An image of Mercurius, a figure symbolising the completion of the alchemist’s work, appears alongside Ball’s commentary on chrysopoeia. In this picture, taken from a Rosicrucian compendium of 1760, Mercurius appears as a three-headed dragon. The first head, a moon with the face of an eagle, symbolises mercury; a second, representing sulphur, takes the form of a cheerful yellow sun; while the third is a sun-moon fusion combining the astrological symbols for mercury and the sun. From the third head protrudes a long beak, grasping the pale tip of the dragon’s tail. The base of the tail is darker, greenish and emerges from a dark blue body that possesses a fourth head, charcoal-black and ugly as a troll’s. There are allusions here to the Ouroboros – the snake that eats its tail, a common motif in alchemical art, signifying the union of opposites – and to the process by which the philosopher’s stone was supposed to be manufactured. For the alchemist’s ingredients to metamorphose into the philosopher’s stone they had to be forced through a series of colour changes: an initial phase, the nigredo, or blackening, was followed by albedo, whitening. In the final phase, rubedo, the stone turned red. Ball, drawing on alchemists’ notebooks and lists of ingredients, makes an educated guess as to what was going on here: lead, when heated in air, forms black, yellow and red oxides. Red lead (lead tetroxide) had been used as a pigment from antiquity.

There are many accounts of the successful fabrication of gold by alchemists, though we can be sure that, at least by the standards of modern chemistry, a transmutation never took place. Ball explains that though there were charlatans who set out to trick people, many alchemists genuinely believed their attempts had worked. For one thing, it was difficult to establish whether what they had produced was real gold. The technique used for verifying gold was an unreliable process known as ‘cupellation’, in which metals were separated from each other by melting and then weighed. If a chunk of putative gold weighed the same after cupellation as it did before, it passed the test. The definition of gold was also unstable. It was widely believed that transmutation could happen in stages – an alchemist could produce a substance that was approaching gold or near enough to gold – and if a metal possessed some but not all the properties of gold, it might be gold but of an inferior sort.

Another three-headed dragon, from a German manuscript of 1738, makes an appearance in Ball’s book. It has a sulphur head and a mercury head, but its third signifies ‘salt’, a category that encompassed a broad variety of compounds, as it does in modern chemistry. It was introduced into alchemy by perhaps the most famous alchemist of the Renaissance, Philippus Aureolus Theophrastus Bombastus von Hohenheim, better known as Paracelsus. He believed that these three ingredients determined the form not only of all metals, but of all matter, including plants and animals. The proportions of sulphur and mercury in a substance determined its combustibility and liquidity, whereas the amount of salt decided its solidity.

Paracelsus’s doctrine of the tria prima informed his work as a physician. Syphilis and periodic outbreaks of bubonic plague had exposed the medical orthodoxy of the time – based on Galen’s theory of the four humours and heavily reliant on bloodletting – as worse than inadequate. Paracelsus, a vicious critic of Galenic doctors (‘a misbegotten crew of approved asses’), transformed the field of alchemy by instructing his peers to make medicines, not gold. Though he was despised by the medical establishment, he became a well-reputed doctor: Erasmus was one of his patients, and he was appointed city physician and made a professor of medicine at Basel. He was the first doctor to prescribe laudanum; he promoted treatments using tinctures and alcoholic extracts; and he is now regarded by many as the father of toxicology. But he was also an occultist and mystic, whose writings are sprinkled with sigils, astrological seals and instructions for kooky charms – a trident made from a horseshoe, for example, intended to restore potency to those cursed by witchcraft. The story of his life was embellished by legend: it was widely believed that he carried the elixir of life in the pommel of his broadsword, rode a magical horse all over Europe to meet with demons, and that he had created a homunculus.

Alchemists have been often written about as failed scientists, crippled by their spiritual beliefs, or as mystical philosophers whose hours at the athanor are beneath notice. Ball, a former editor at Nature and a biographer of Paracelsus, rescues their scientific reputation while recognising the importance of the mystical philosophy that informed their work. This philosophy can be tricky to pin down: partly because it changed shape, especially during the Renaissance, when translations of Neoplatonist and Gnostic texts from Greek and Arabic appeared, and partly because it differed from alchemist to alchemist. But at its core were the teachings of Hermes Trismegistus, a mythic composite of the Greek god Hermes and the Egyptian moon-god Thoth. The most popular source for Hermes’ teachings was the Emerald Tablet, a cryptic text which spread through Europe after it was translated into Latin in the 12th century. Its best-known line (‘What is below is like what is above, and what is above is like what is below’) encapsulates the Hermetic view of reality, in which the microcosmic mirrors the macrocosmic: smaller systems reflect larger systems, at every scale. Chemical reactions reflect human dramas, which reflect celestial movements, which reflect the mind of the divine. Hermes goes on: ‘All things came from the One … All things are born from this One by adaptation … This is the force of all forces, for it overcomes all that is subtle and penetrates solid things.’ The alchemist’s lab work and the philosophy were inextricable: working with material substances was a sacred endeavour, because matter and spirit were modalities of the same substance.

Books of alchemy constantly draw out the connections between worlds. The heavenly bodies each have a corresponding metal: the Sun gold, the Moon silver, Mars iron. The various states of the philosopher’s stone are represented by animals: the red dragon of rubedo, the white eagle or dove of albedo, the green lion devouring the sun to bring about nigredo. Paracelsus devised a ‘doctrine of signatures’, according to which medicinal herbs and minerals resembled the parts of the body they could be used to heal – heart-shaped leaves were good for the heart, and so on. In the thick of these endless correspondences, the alchemist saw himself operating on multiple levels of reality. By examining the behaviour of metals, he hoped to gain insight not only into their material properties but into the most profound mysteries of the universe.

The belief in the interrelation of the various planes of reality formed the basis for what came to be known as ‘spiritual alchemy’. This is the idea that alchemical transformations correspond to spiritual transformations in the alchemist, and that the real goal of alchemy is not the creation of the philosopher’s stone but the realisation of spiritual enlightenment. Alchemical texts often seem to be hinting at this inner work. The Compound of Alchemy, written in the 15th century by a canon of Bridlington called George Ripley, describes the twelve ‘gates’ of purification through which the stone must pass if it is to be successfully manufactured, or – reading between the lines – through which the soul must pass to be successfully redeemed. Metaphors of ‘body’ and ‘spirit’ are used so prolifically that the line between metaphor and referent becomes blurred: Ripley instructs his reader to ‘make a marriage the Body and Spirit betwixt’ and writes of the ‘Cojunccion made of thyngs three,/Of Body, Sowle, and Spyrit tyll they not stryve,/Whych Trynite must be brought to perfyt unyte’. This is not just a recipe for lead tetroxide.

Acrucial influence​ on spiritual alchemy was the Lutheran mystic Jakob Böhme, a shoemaker from Görlitz who had a powerful experience of divine revelation in 1600 as he gazed into a beam of sunlight reflected in a pewter dish. Böhme believed that in this moment God had clarified the spiritual structure of the world, and the nature of good and evil. In his books – the first of which, Aurora, was written twelve years after the vision – Böhme used alchemical language to describe the stages by which the soul could be restored from its fallen state into one of divine grace. His ideas were adopted by the Rosicrucian movement, which aimed to guide humanity’s spiritual evolution and bring about a new, more harmonious era. By the end of the 17th century, alchemists had divided into two irreconcilable camps: the toilers in the lab on one side, and the Rosicrucians, who believed that ‘godless and accursed gold-making’ was less important than the work of inner purification, on the other. As practical alchemy was subsumed into science, spiritual alchemy became the prevailing form. It was a component of the occult revival of the late 19th century: the Hermetic Order of the Golden Dawn incorporated spiritual exercises based on alchemy into its training programme, which aimed to dissolve the false persona and release the initiate’s true self. In the 1940s, Carl Jung co-opted spiritual alchemy as a metaphor for ‘individuation’: the repressed, mercurial unconscious would combine with the dominant, sulphuric consciousness to produce a unified self.

It’s unclear whether spiritual alchemy was any more successful in purifying the spirit than lab alchemy was in fabricating gold, but in pursuit of its goal, alchemy – both alchemies – deposited real treasure. The greatest books of alchemy contain exquisite illustrations alongside gnomic writing that, in its opacity and richness of imagery, resembles a kind of proto-Symbolism. Ball includes plates from one of the most celebrated books, Atalanta Fugiens, composed in 1617 by the German alchemist Michael Maier. An engraving entitled Nutrix ejus terra est (‘the Earth is its nurse,’ a line from the Emerald Tablet) depicts the Earth as a woman with the globe as her body, nursing a child, while at her feet a she-wolf suckles Romulus and Remus, and baby Jupiter is suckled by the goat Amalthea. On the opposite page, Maier has printed the tune and lyrics to a hymn of his own composition, which declares the nurse of the wise man, the source of his wisdom, to be the Earth herself. Another dazzling alchemical text, the Ripley Scroll – of which there are several known versions dating from the early 16th century to the mid-17th century, some of them twenty feet long – prints The Compound of Alchemy alongside ornate, symbolic imagery. A section on the ‘Red Sea’, for example, represents ‘mercurial water’ as menstrual blood raining from the body of a dragon with a crescent moon between her jaws. Perhaps the most beautiful alchemical book of all, not mentioned by Ball, is the Splendor Solis, an illuminated manuscript from the 16th century illustrating a text attributed to Salomin Trismosin, a teacher of Paracelsus. Many of the brilliantly coloured images from this work are representations of specific alchemical processes, in which a large glass flask containing symbolic creatures appears at the centre of the page, while outside the flask, detailed scenes from life play out under astrological portents. One page shows the flask with a peacock inside. Above the flask, Venus appears in her chariot pulled by turtle doves, while below, people make merry: a band plays music, a group sits at a table drinking wine, lovers embrace or stroll on the grass.

Alchemical art and literature was influential even in its time. Hieronymus Bosch borrowed from the pictorial language of alchemy, stuffing his Garden of Earthly Delights with its instruments and symbols: eggs and pelicans, glass tubes, a young couple pleasure-cruising in a glass sphere. The closed doors of the triptych show the Earth inside a transparent globe, with clouds condensing at the top of it. His Adoration of the Magi also alludes to alchemical themes: a small gold sculpture of the sacrifice of Isaac rests on three black toads, symbols of nigredo, while the buildings in the background look unlike those in any Dutch city but resemble contemporary drawings of the athanor. These references would have been recognised by many of Bosch’s audience, some of whom would have had experience of the tools and symbols of alchemy. Whether Bosch wanted to convey some kind of Hermetic-Platonic subtext, or playfully suggest it, or whether like van Helmont he simply enjoyed the shape and shimmer of alchemical equipment, we can’t be sure.

The Surrealists, similarly fascinated with psyche and symbolism, saw themselves as the alchemists’ successors. André Breton, in his Second Manifesto of Surrealism, pointed to the ‘remarkable analogy … between the surrealist efforts and those of the alchemists’. Ball notes the influence of alchemy on the work of Max Ernst, many of whose pictures feature alchemical symbols and apparatus, and who was compared by Breton to the ‘arch-sorcerer’ Cornelius Agrippa. In 1976, Salvador Dalí produced Alchimie des philosophes, his attempt at an alchemical tome in the tradition of Atalanta Fugiens or the Splendor Solis, consisting of ten alchemy-themed prints which were concealed in a box resembling two giant hidebound manuscripts. Its images of the sulphuric king and mercurial queen in blue and gold crowns, a peacock with a woman’s body in the flames of a furnace, the snakes of the caduceus next to Adam and Eve (seemingly a direct reference to a picture from a Ripley Scroll), attest to a serious engagement with alchemical texts and symbolism.

But it’s in the work of the British Surrealist and member of the Hermetic Order of the Golden Dawn, Ithell Colquhoun, that the marriage of alchemy and Surrealism finds perhaps its fullest expression. Her paintings, the subject of an excellent recent retrospective at Tate Britain, replay over and over the dialectic of mercury and sulphur. In The Sunset Birth (c.1942), two megaliths, one fiery, one watery, engage in a tug of war: tendrils of blue and orange energy shoot out from the stones to twine around each other; a third stone – a red toroidal lapis – hangs between them. Pushkin Press has reissued some of Colquhoun’s books, including her novel Goose of Hermogenes (1961), which is another term from alchemical literature for the philosopher’s stone. The chapter headings refer to Ripley’s twelve gates – ‘Calcination’, ‘Solution’, ‘Separation’, ‘Conjunction’ etc – but the story of its protagonist’s escape from her uncle’s island, where she has been imprisoned, is told in far from linear fashion. The novel is a fusion of Surrealism, psychoanalysis and adventure story, in which alchemy features as both a language for describing liberation from oppression, and a source of powerful oneiric effects: ‘I open the veins of my arm with the cut of a sliver of silicon. Blood pours out from the left … circling the island a ribbon of stain in the foam unmixing like a rusty chain to bind him in binding … of mercurial metal.’ In Colquhoun’s hands, the trade secrets of ancient Egyptian metallurgists are realised as visionary literary modernism.