First of all, Script is a stacking programming language in which Bitcoin works. Script is pretty basic and limited for security reasons. For this same reason, Script is not Turing complete.
Why does the bitcoin protocol opt for the use of this script-derivative in the transaction instead of just using the output address plainly?
This is because locking bitcoins up with a script (also known as a smart contract) allows a wide variety of conditions in which specific Bitcoins would be spendable (multi-signature, a date in the future, a simple mathematical puzzle, etc).
There are some standard transactions in Bitcoin:
P2PKH (Pay to Public Key Hash)
P2SH (Pay to Script Hash)
P2WPKH (Pay to Witness Public Key Hash)
P2SH_P2WPKH (This is done to make P2WPKH backward compatible.)
P2WSH (Pay to Witness Script Hash)
P2SH_P2WSH (This is done to make P2WSH backward compatible.)
The script allows to tell the network how those bitcoins can be spent.
So, in the case of a P2PKH transaction, you are locking the Bitcoins to a certain public key which is represented by the address. The address is the base58 representation of the hash of the public key, and only the person who possesses the private key of that public key can spend those Bitcoins.
However, what really exploits the potential of the Script language in Bitcoin, is the standard P2SH. Here, you are locking the funds to a Script program (or smart contract) instead of a public key. So whoever can satisfy the problem in that program can unlock the funds and spend them. It is worth to note that there is a standard inside this standard. A P2SH can be standard or nonstandard depending on certain conditions. The major cases of use for P2SH transactions, however, are multi-signature transactions, and SegWit backward compatibility transactions (P2SH_P2WPKH, P2SH-P2WSH).
P2WPKH and P2WSH are SegWit standards which aim to reduce the fees in transactions, but the principles from above remain (P2PKH, P2SH).