UOM Conversion Factor error in PO

I have two items with different conversion factor for a single unit. For eg: I have apple and orange in stock list. My default UOM is kg and purchase UOM is box. Conversion factor is 6 for Apple and 8 for Orange. When I created a purchase order, I selected apple first, so system is showing the UOM as box and conversion factor as 6. Then I removed the apple from item code and selected orange. But the system is still showing the old conversion factor. it is not taking the conversion factor as 8 for the oranges. This is creating lot of issues for our clients as they are dealing with multiple UOM and different conversion factor.