COUNTIF formulas do not calculate in excel until you double click the formula and hit enter
You need to explicitly evaluate the formula in POI after changing it, or the values, otherwise you need to load it in excel and visit the cell to trigger a re-calculation. See http://poi.apache.org/hssf/eval.html
The CountIf function has not yet been implemented in the eval section. This bug needs to stay open till an implementation is provided.
COUNTIF() implementation was added in bug 44345.