Most people knowledgeable about precious metals recommend that the best way to buy them is to have physical possession of them. There are many possibilities here; you can have them in a safe in your house, under your mattress, hidden somewhere in your house, buried in your yard, with a friend, or in a safety deposit box at a bank.
Safe deposit boxes do have some real drawbacks. For example, they typically are not insured (however, the chances of getting your safety deposit box broken into is probably much less likely than your home being robbed). Also, some states will seal your safe deposit box upon your death. And the I.R.S. can freeze your box, preventing you from being allowed to open it.
A safe deposit box at a bank is usually very safe. However, there is a lot of questionable information and rumors out there, so lets address them.
If there is any question, Executive Orders are written for an entire country, not individuals -- and the hoax refers to individuals (e.g. stating that the government knows that they have bullion).
We have carefully examined the real Executive Order 6102 and writings of the time, and nothing in it even suggests that safe deposit boxes were sealed. Sealing every safe deposit box in the country would have been a massive undertaking, and the manpower to have an IRS agent present at every opening just wasn't there (there was about 1 IRS agent for every 5,000 individuals).
Ironically, the first version of the Wikipedia article on Executive Order 6102 was just the text of the hoax. Later, it was referred to as a rumor, and the entire text quoted. Then when people started trying to get the text of Executive Order 6102, they would quote the text of the hoax (as the Wikipedia article contained the full text of the hoax, but not the real text), spreading the hoax further.
The clearest evidence that this is a hoax is that it refers to gold and silver, although silver ownership was not banned in 1933!
First, cases like this are very rare. Second, none of this applies to bullion. If you have a 1-ounce gold bar in your safe deposit box, and it is accidentally opened and auctioned off, you will get the proceeds. The bar isn't going to be sold for significantly less than the spot price, and shouldn't hold sentimental value to you.
A guy in a suit going by the name 'doobsta' on YouTube at www.youtube.com/watch?v=SuvzTwFZlFM states that U.S. legislation states that 'During a bank holiday, the U.S. Government has the right to open up all safety deposit boxes', and that there is a list of items that they can confiscate, including precious metals. However, this term only appears on two websites, both pointing to that YouTube guy. And the Patriot Act does not mention bank holidays.
Another story at bellaciao.org/en/article.php3?id_article=10012 claims that in the event of a 'national disaster', only agents of Homeland Security will be allowed to open safe deposit boxes, and that 'no weapons, cash, gold or silver will be allowed to leave the bank' and 'only various paperwork will be given to the owners.' This sounds very suspicious. First, 'allowed to leave the bank' suggests that the items would not be confiscated (that they would have to be put back in the box). Second, it covers what IS NOT allowed, and what IS allowed -- but leaves out lots of stuff. What about trinkets? Jewelry? Stock certificates? Bearer Bonds? Platinum bars? Government lawyers would never have a list of things that are included and not included -- they always either list what is included (with everything else being excluded) or everything that is excluded (with everything else being included).
In the event of a true state of emergency, the President could come up with an Executive Order that would handle all of this -- so there is no need for a top-secret safety deposit box procedure that 10,000s of bank employees would be told about (during a 2-day workshop, no less!).
They cannot simply go to the bank and open up your safe deposit box. However, if they have a judgement against you, it sounds like they can freeze your account (where you would be unable to open the box), using a 'Notice of Levy'. Then, you would be present during the opening of the box.
As an example, see this document from the California Courts.
There are some people that may imply otherwise. However, there is nothing restricting you from having cash in a safe deposit box.
I have not heard of any banks that insure the contents of safe deposit boxes (think about: how would they know how much insurance to purchase unless you told them how much the contents of your box was worth?).
However, it is possible to get insurance for the contents of safe deposit boxes. Your homeowners/renters insurance is one way, but they would normally require a list of specific items in the box. The bank may also offer insurance. Another way is through the private company Safe Deposit Box Insurance Coverage, LLC (a/k/a SDBIC, not to confused with FDIC or the defunct FSLIC, both government agencies). They offer insurance coverage for safe deposit boxes (using insurer AXA Art), covering any legal contents of the box up to the coverage limit. No listing of contents or appraisal is required. Besides burglary, it also protects against other events (such as hurricanes or floods). Sample prices I saw (in 2018) were $50/year for $10,000 coverage, $110/year for $50,000 coverage, $500/year for $250,000 coverage (roughly 0.2% of the value per year).