What is the purpose of a stackmachine if only some standard script forms are allowed? And why are some scripts allowed and perfectly fine in mined blocks but rejected by clients if found in transactions; what's the point of that? The more I read about script the less it makes sense.
OK, the original concept of scripts was not perfect. But with p2sh (BIP-16) any script can be realized
The bitcoin script mechanism is very simple and dumb, that's the reason why it is so secure. the more it will be complex the less it can be secured cause more scenarios can occur and points of failures. so you have a very simple mechanism with limited operations, very easy to test and maintain securely, still extendable such as pay to script hash.
Bitcoin was the first crypto coin. Satoshi when analyzing the problem of digital payment discovered that the machine needs to determine if some transaction sucessfully or unsuccessfully spent the output. In the most trivial case this means the signature is ok and pubkey matches.
Determining if anything (the spend) is one of two possible values (1=success, 0=failure) is the most studied problem in computer engineering, also known as set membership or recognition of word from language.
Standard off-the shelf solution to this problem is to use some machine that meets the criteria. For bitcoin the requirement that the machine should always halt, as fast as possible. the strongest machine with this guarantee is some kind of a stack machine.
TL;DR Satoshi just used the obvious solution to a known subject
Well the reason for allowing only some script forms is to ensure that the language is not Turing complete. The simplest reason why you cant have a Turing complete language is because you want a block to be pushed up into the blockchain every 10 mins. So you have to find a way to indirectly control execution time. Certain other scripts are prevented because of feared security vulnerabilities. A Turing complete language within bitcoin would call for a solution of the kind implemented by Ethereum.
Your second point about why certain scripts are allowed in blocks but not in transactions is an interesting one and is mostly because some of the vulnerabilities were discovered well after bitcoin got rolling. So invalidating the blocks that had the problematic opcodes would have required a hard fork. Whereas just blocking the problematic opcodes to prevent future issues was much easier. Now it would be interesting if redeeming the coins in some block necessitated the use of any of the blocked opcodes, but I guess there aren't any transactions of that kind.