If I delete the Bitcoin wallet from my HD, my Bitcoins will be still alive inside the Bitcoin network, right? Is there any way to remove my bitcoins from the network in a permanent way?
If you don't use them, nobody will be able to use them, so in a sense they will have disappeared. If you wish for them to be publicly, permanently destroyed, you could try one of the approaches at What is the Bitcoin equivalent of /dev/null?, such as sending them to the address 1111111111111111111114oLvT2, which is the address of an all-zeroes hash. There are technically many private/public keys that correspond to the address, but finding one is practically impossible.