The results, based on brain scans of 60 people, mean the drug could treat depression in a unique way, the researchers say.
Psychedelics are being studied to treat a range of mental health disorders.
Patients with depression are warned not to take psilocybin on their own.
A synthetic form of the drug is tested on people in trials under strict medical conditions, with psychological support from experts provided before, during and after it is taken.
Prof David Nutt, study author and head of the Imperial College London's Centre for Psychedelic Research, said the latest findings on psilocybin were "exciting" and "important".
With depression, the brain can get stuck in a rut and locked into a particular negative way of thinking, he said.
But when given psilocybin, people's brains opened up and became "more flexible and fluid" up to three weeks later.
This could be seen in increased connections between regions of the brain when patients were scanned. These patients were more likely to experience an improvement in mood months later.
Similar changes were not seen in the brains of people treated with a standard antidepressant.
"This supports our initial predictions, and confirms psilocybin could be a real alternative approach to depression treatments," Prof Nutt said.
Psychedelics are a type of hallucinogenic substance which affects all the senses, altering a person's thinking, sense of time and emotions.
While regular antidepressants are taken every day, psilocybin may only need to be taken once or twice to produce the same effect - but further research on more patients for longer is needed to confirm that.
The results, published in Nature Medicine, are taken from two studies. In the first, everyone received psilocybin; and in the second, a randomised controlled trial some were given the drug while others were given a different antidepressant.
All participants also received talking therapies with registered mental health professionals. Brain scans were taken before, and then one day or three weeks after taking the therapy.
Prof Robin Carhart-Harris, senior study author, said: "We don't yet know how long the changes in brain activity seen with psilocybin therapy last, and we need to do more research to understand this.
"We do know that some people relapse, and it may be that after a while their brains revert to the rigid patterns of activity we see in depression."
Earlier findings from the studies showed a reduction in symptoms of depression with the psilocybin therapy, but the researchers were not sure how and why it worked.
Now they want to test their theory of changes in brain connectivity on other mental health illnesses, such as anorexia.