Ok so there is no actual needing of "hooksecurefunc", it also needs a /reload to "unhook" so I don't think it's a good idea to use that command, but I may be wrong..

By the way, this is the script I wrote, tested, working. Very basic script, it stops you from casting Psychic Horror on a target with Cloak of Shadows buff.

local spellnametoblock = "Psychic Horror"
local spellidtoblock = 64044 -- Psychic Horror
local bufftoavoid = "Cloak of Shadows"

UseAction = function(slot, tar)
  -- print("Slot: "..slot)
   --print("Target: "..tar)
local type, id, subtype, spellID = GetActionInfo(slot)
	if type == "spell" 
	then
	--print("Attempting to Cast Spell ID: "..spellID) -- Get the ID of the Spell you want to block
		if spellID == spellidtoblock
		and UnitBuff("target",bufftoavoid)
		then
		print("You can't cast "..spellnametoblock.." if Target has "..bufftoavoid.."!")
		else
		CastSpellByID(spellID,tar)
		end
	elseif type == "macro"
	then
	--print("Macro ID: "..id)
		local macrospellname, macrospellrank, macrospellID = GetMacroSpell(id)
			if macrospellname == spellnametoblock
			and UnitBuff("target",bufftoavoid)
			then
			print("You can't cast "..spellnametoblock.." if Target has "..bufftoavoid.."!")
			else
			RunMacro(id)
			end
	end
end

You just need to change the first 3 lines, put the exact name of the spell to check, its ID (there is a print option to get the ID once you press the Spell) and the debuff to check.
The reason because you need both spell name and ID is because this is the onyl way to make it work with both normal spells and macroed spells.

There is only one problem at the moment: not tested but pretty sure it can't prevent you from casting a macroed spell on a custom unit. For example, if you have macroed Psychic Horror on Focus, and Focus has CoS, and you press that macro, script won't work. I don't think there is a way to see what is the target of your macroed Spell.
And viceversa, if you want to Psychic Horror your focus who has no CoS, but your target HAS CoS, the script will probably stop the spell, because it "sees" you are trying to cast Psychic Horror on a CoS target.
So yea the macro part it's kinda tricky, not sure if it's possible to add all these checks and make a perfect script.

Well my main goal was achieved, not complaining, and this idea was funny to do...but not sure if I will work on it again, because of all these limitations, I don't see the potential in LUA to make a perfect script actually.

 

Great post. Thank you for sharing your findings!!