Not rude, not stupid and never an idiot.
It is true there are many people who make fun of Americans trying to speak Spanish, but there are many more who appreciate sincere efforts to speak it.
I don't know how it is in the USA, since opinions are so divided about these issues, but I guess that if the Latino you meet has a foreign accent in English, they may be happy to speak Spanish with you.
I understand how insecure one feels speaking a foreign language (that happens to me all the time), but try and feel comfortable; you'll be fine.
