Ghosts behave like Wraiths, hovering above the ground and passing through solid tiles to reach the player. In order for Ghosts to travel up to players, they must have a solid block to pass through, or else they will stay hovering just above the ground.
When a Tombstone is broken, there is a 50*1/2 (50%) chance it will cause a Ghost to spawn if there are already less than 2 Ghosts active in the world, and any of the following conditions are met:[1]
Ghosts can be attacked through walls with melee weapons that cannot usually hit through walls – this makes spears and whips especially effective. Both are still great for dealing with large hordes of Ghosts, because they can hit many of them at once.
In Journey Mode, even when the enemy spawn rate is set to 0× the normal rate, mining up Tombstones may still cause Ghosts to spawn.
In multiplayer when another player dies, a ghost can spawn near a living player.
Trivia[]
Ghosts are the only pre-hardmode enemies who have that death sound.
The Bestiary entry for the Ghost: "During the most harrowing times, these apparitions float aimlessly through the depths to steal the life from those they envy."
Desktop 1.4.0.2: Ghosts are more likely to despawn, and the conditions which allow them to spawn has been reduced as well as the maximum number of them.
↑Information taken from the PC1.4.2.3 source code, method CheckSign() in Terraria.WorldGen.cs. There may be inaccuracies, as the current PC version is 1.4.4.9.