This is on 3.0.1m15 Build:9379 on Ubuntu 8.10 and package from
http://ppa.launchpad.net/openoffice-pkgs/ubuntu

1. create new writer document
2. options: autocaption for pictures is ON
3. insert new non-square image
4. select picture from image context menu
   size is preselected as:
   width 100% (relative ON)
   height 11,46cm (relative OFF)
   keep ratio ON
 a) press RESET -> width=0cm
    press RESET -> width=16cm
    press Cancel
 b) Go back to picture menu
    unselect width relative -> width=16,46cm, height=16,46cm
    select width relative -> width=100%, height=16,46cm
    select height relative -> width=74%, height=100%
    unselect height relative -> width=74%, height=12,18cm
    unselect width relative -> width=12,20cm, height=12,20cm

... mix even more with select/unselect, original size and reset and you can see
every imaginable value on earth appearing in the size boxes - but nothing expected.

This may be related to #63028, #77402, #86978, #96064 and #89459