The rules as written are vague on when spellcasting is noticeable. According to how I read the rules, the EFFECTS of spells may be undetectable, but the ACT OF SPELLCASTING is usually somewhat obvious.
From PHB p. 205: “Unless a spell has a perceptible effect, a creature might not know it was targeted by a spell at all. An effect like crackling lightning is obvious, but a more subtle effect, such as an attempt to read a creature’s thoughts, typically goes unnoticed, unless a spell says otherwise.”
Compare that rule on effects with the note about spell components on PHB p. 203. The rule makes it clear the spellcaster is doing something obvious when attempting the spell: Verbal (V) need the “chanting of mystic words” and Somatic (S) involves “forceful gesticulation or an intricate set of gestures.”
Also, sorcerers have a specific class ability to cast on the sly, the “Subtle Spell” metamagic costing a Sorcery point, inferring those without this ability must be noticeable when casting a spell.
As a house rule, when a character wants to cast a spell and not draw attention, use these guidelines:
* If a spell has a verbal component, the caster must make a Charisma (Deception) check greater than observers’ Passive Perception, else others notice strange overtones and eldritch words uttered from the caster.
* If a spell has a somatic component, the caster must make a Dexterity (Sleight of Hand) check greater than observers’ Passive Perception, else others notice glowing glyphs, sparks or other strange lights twinkle around the caster’s twisting fingers.
* The Invisibility spell grants advantage to these checks just as the caster becomes visible again. The Greater Invisibility spell allows the caster to remain Invisible even while casting spells.
Note that noticing spellcasting does not mean observers know which type of spell is being cast. That step requires an Intelligence (Arcana) roll as an automatic free action…
* 9 or less … Spell not recognized.
* 10 to 14 … Spells of 1st to 3rd level recognized.
* 15 to 19 … Spells up to 5th level recognized.
* 20 to 24 … Spells up to 7th level recognized.
* 25 or more … Spells up to 9th level recognized.
* Sorcerers using the “Subtle Spell” metamagic ability automatically succeed.
* If a spell is cast as a ritual, it is always obviously a casting attempt.