So, you cannot touch the ball with your arms unless they are by your side? If you intentionally hit the ball by sticking out your elbow that is a handball?
It depends on a lot of factors, and often its up to the discretion of a referee.
For example, it matters how close the player is to the opposing player who shoots the ball. If an opposing player is very close and shoots the ball very hard in my direction, and I obviously cannot get my hand (or arm) out of the way fast enough, a referee will probably not call it a handball (especially if its in the penalty area). Thats because its not fair to the player who cannot get his hand out of the way fast enough, but also because the opposing player may have tried to shoot at my hand on purpose to draw a penalty.
But if you're at a fair distance, and there's a cross or shot on goal, and you then raise your hand and it hits your hand, its almost always a penalty (or handball foul). This is especially true if the ball was headed towards goal, or if it was a cross to a player in the box who was in a scoring position.
But as a general rule, if you keep your arms down and close to the body, and the ball then hits your hand or arm, its never a penalty or handball foul.
They call that "ball to hand".
And BTW, your arm is just as guilty as your hand. Both body parts are considered handball fouls