From what you describe, I'm guessing that what you are looking for is something like Casascius coins. These display the public key but conceal the private key behind a tamper evident holographic sticker. This means that when you pass a physical coin to someone, they know that you don't have the private key, so they are the sole owner of the bitcoins at that address.
If this is what you are looking for, there are a few problems to bear in mind:
- The manufacturer of the coins puts the private key behind the sticker, so you are trusting that they did not also keep a record of the private key.
- You are trusting in the effectiveness of the tamper evident sticker.
- Since November 2013, Casascius no longer sells these coins, although there are many in existence sold before that point.
The first point can be solved by using 2 factor authentication, so that the string that the manufacturer hides behind the sticker is useless without a password that is never shared with the manufacturer.
So what you require is possible, although sidestepping the blockchain does introduce trust in the tamper evident nature of whatever method you use to conceal the private key. If the value stored by this method is sufficiently small per item to make the level of trust acceptable, then this may not be a problem.
Tamper evident method without the physical form
If you are looking to use this approach without the physical form, then I can't see a way of keeping ownership to just a single person without either a decentralised system that is effectively a parallel version of the block chain, or a centralised system where you have to place trust in those running it.
The solution to this problem of trust is the block chain - the public distributed ledger. If you want to avoid using the block chain, then you are going to have to live with the trust problem unless you can invent something new.
If the underlying reason for asking this question is that you want to conduct transactions anonymously, there are a variety of ways of doing this which you can learn about by browsing the anonymity tag.