There are many plants that are safe for you to keep in your home. Aloe, for example, is quite popular, and safe. Eucalyptus has a nice scent and budgies even eat these in the wild! Also safe are gardenias, african violets, bougainvillea, chamomile, garlic, magnolia, parsley, grape vine, roses, petunias, peppermint and hibiscus.
If in doubt, do a google search for the plant you want to buy and add the words "safe for birds". You should find a list of websites that will tell you whether the plant you want is safe or not! :)
When in doubt, always err on the side of caution. If you're not sure, don't put it in your home where your budgies can get to them. But with so many plants safe, there's sure to be something you like that's not toxic for your birds!