If I delete the Bitcoin wallet from my HD, my Bitcoins will be still alive inside the Bitcoin network, right? Is there any way to remove my bitcoins from the network in a permanent way?
If you don't use them, nobody will be able to use them, so in a sense they will have disappeared. If you wish for them to be publicly, permanently destroyed, you could try one of the approaches at What is the Bitcoin equivalent of /dev/null?, such as sending them to the address 1111111111111111111114oLvT2, which is the address of an all-zeroes hash. There are technically many private/public keys that correspond to the address, but finding one is practically impossible.
If you delete your wallet and have no other copies around, this means those bitcoins are gone forever because no one has the private keys to move them.