Issue 35178 - Data are lost from user's view in mail merges based on spreadsheet data
Summary: Data are lost from user's view in mail merges based on spreadsheet data
Alias: None
Product: Base
Classification: Application
Component: code (show other issues)
Version: OOo 1.1.3
Hardware: PC All
: P3 Trivial with 5 votes (vote)
Target Milestone: OOo 3.0
Assignee: h.ilter
QA Contact: issues@dba
Keywords: oooqa
: 93858 (view as issue list)
Depends on: 58882
  Show dependency tree
Reported: 2004-10-08 10:29 UTC by tuharsky
Modified: 2010-06-22 10:51 UTC (History)
5 users (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---

The sample Spreadsheet (26.26 KB, application/vnd.sun.xml.calc)
2004-10-08 10:32 UTC, tuharsky
no flags Details

Note You need to log in before you can comment on or make changes to this issue.
Description tuharsky 2004-10-08 10:29:36 UTC
User has his data stored in Spreadsheet file. When he does the MailMerge on text
document using data from the Spreadsheet source, some data are sompletely
ignored by OOo!

Try this:
Add the spreadsheet to the Data source. Then create any text document and place
some fields pointing to the source.
Now try MailMerge. Even in the MailMerge window You can see, that some values
are simply missing -the cells are blank, even when this isn't true in the
Spreadsheet! For example look at column AR named Vymer1, that is full in all
rows of Spreadsheet. In the MailMerge, many of cells are simply blank!

If You do the MailMerge against these facts, You cannot be surprised that in
generated documents the data are also missing!

The bug is tested in OOo112-sk/w2k and OOo113rc-sk/Linux, so I set OS to All.
Comment 1 tuharsky 2004-10-08 10:32:16 UTC
Created attachment 18227 [details]
The sample Spreadsheet
Comment 2 tuharsky 2004-10-08 10:42:38 UTC
The bug also affects OOo1.9m49/Linux and OOo1.1.0/Linux. I'm going to download
the 1.0 series and test it, cause the problem is quite critical for us.
Comment 3 tuharsky 2004-10-08 12:19:30 UTC
It seems, that the bug is connected to the slashes in the text. The missing
cells in columns Vymer1, Vymer2, Vymer3 are those with slashes inside. Other
words, there are no cells containing slashes visible in the Mail Merge window in
Vymer1, 2, 3 columns.

Although column AH named Cislo uctu dlznika contains alltogether numbers and
slashes and it is displayed OK.
Comment 4 tuharsky 2004-10-08 12:27:26 UTC
The other thing is strange: the last two rows (78 and 79 wit "Por" numbers of 77
and 78) are completely missing in the MailMerge window. Don't know, if this is
another bug or the same.
Comment 5 tuharsky 2004-10-08 12:51:20 UTC
More accuracy:

If user generates output using Print function, the -let's call them "discarded"-
cells are presented by "0" in output files or print. We see here typical action
of  bug 34992 (placing "0" where blank cell in MailMerge appears).

This could mean, that the cells are really "blanked" when opened in MalMerge.
Thus the bug may be somewhere in import functions of Database Access?

Btw. I wonder: if the bi-directional data update functioned already (as planned
for future OOo), would the cells be blanked in the source file also, or not? I
mean: If the bug would creep into OOo 2.0, wouldln't it might cause even much
more damage by overwriting even source data?
Comment 6 tuharsky 2004-10-11 08:00:30 UTC
I have tested replacing the "/" character by "\", "-", " " and "m" but no effect.
Comment 7 tuharsky 2004-12-08 14:32:33 UTC
Issue 31472 might be connected somehow.
Comment 8 thackert 2005-01-09 15:53:18 UTC
I have seen a lot of entries, but no comment from any OOo-developer or someone
else since December 8th ...
@tuharsky: have you tried your sxc file with a newer version of OOo like 1.1.4
or a newer version than 1.9m49 (like 1.9.m65 or 1.9m69)? Is the problem still there?
Comment 9 tuharsky 2005-01-10 10:54:00 UTC
In 1.1.4, bug still present.

For 680, I'm downloading the RPM files, I have problems installing them since
I'm using Debian, however. If I'd be successfull, I'll surely let You know.

OT: Someone should make installation process more generic and simple for non-RMP
distributions. I know there's a howto, but it's quite silly to do all the
hacking just to install an office suite in year of 2005.. Binary installator
worked very well, the switch to RPM was unpleasant from my point of view.
Comment 10 tuharsky 2005-01-14 08:39:41 UTC
With OOo1.9.71 build-3 sk, BUG IS THERE.

Because the Mail merge is completely reworked for 2.0, I wonder, if the bug
should be marked "Blocking 2.0"..
Comment 11 Frank Schönheit 2005-01-14 09:21:48 UTC
When accessing a spreadsheet document as database, the driver needs to guess
which data type each column has. It does so by looking into the first few rows.
In the sample spreadsheet attached to this issue, this, for instance, results in
a numeric type for column Výmer1. Now when in later rows, values *other* than
numeric values are encountered, the driver tries to force them to numerics,
which fails.

A workaround could be to insert an additional first line into the document,
where all cells contain some text which cannot be mis-interpreted as numeric value.

fs->nn: I am not sure whether you already have an issue for this, I seem to
remember darkly that this "auto guess the type" problem has already popped up in
the past ... Not that I'd have a convincing solution for it ...
Comment 12 nathanson 2005-01-29 21:56:48 UTC
I've found that when using the given test spreadsheet in the Mail Merge and Data
Sources screen it makes two tables, "List1" and
"Excel_BuiltIn__FilterDatabase_2". "List1" contains all 78 rows, while
"Excel_BuiltIn__FilterDatabase_2" doesn't have the last two.
Another interesting fact is that if you convert the spreadsheet to a CSV file
using "Save As..." and import that as a Data Source, it gets all the numbers up
to the "/" character (ex: 1620108041/103877 from Vymer1 in row 75 appears as
1620108041), but if you look at the file as a spreadsheet the column data is
intact. Converting the test spreadsheet to a dBase file is similar, but it
changes all the blank cells to "0.00", and adds ".00" to all the other numbers
as well.
All the missing info in Mail Merge is missing if you do queries on the data in
the "Toole > Data Source" dialog as well.

This was all discovered in OOo113/winXP.
Comment 13 niklas.nebel 2005-05-03 13:56:27 UTC
The detection of the column type works as intended (it uses the column's first
non-empty cell). I'm keeping this as an enhancement issue for an improved column
type handling. Suggestions are welcome.
Problems with csv or dbf files are separate and don't belong to this issue.
Comment 14 tuharsky 2005-05-03 14:58:25 UTC
I cannot agree with marking this as "enhancement". The MailMerge obviously acts
differently than any user excepts. It drops values without any notion -fixing
this  clearly means "fixing a bug" for users, whatever it means for developers.

Also the priority should remain P2, since the behaviour leads to unexpected and
sort-of-random data loss in generated mailmerged documents.

The only thing that is correct somehow is the "OOo later" -I can understand
there is huge lack of developers and thus nobody is to fix it. Setting a
milestone surely is in developer's competence, whatever the users demand and
need (however if developer's and user's needs are very different, then the
project is too aimed to developers and too geeky for pure users).

The milestone change thus is the only change of status I can agree with. Please,
let be straight and truly and mark the bug in the way it deserves.
Comment 15 Frank Schönheit 2005-05-10 08:08:28 UTC
fs->nn: I have to admit that I agree to tuharsky here. From the user's point of
view, this is clearly a data loss. The (very good and valid) technical
explanations we have for it do not make a difference to this, IMO ...
Comment 16 tuharsky 2005-12-06 09:42:36 UTC
The issue might be connected with
Comment 17 tuharsky 2006-02-06 13:36:50 UTC

data aren't lost just "from user's point of view". The data ARE in spreadsheet,
and AREN'T in mailmerged document, with no warning or indication of problem.

They are LOST in mailmerged document (period). If user rely on the mailmerged
document (and we have no reason to expect him not to), he faces real data loss,
not a "subjective, questionable one".
He might sended letters, and 1/3 of them could have missing address or postal
code. They will not arrive. He could have stored account numbers into a
spreadsheet. They are lost in output documents. He could have sent thousand of
documents, much of them are invalid, and he even don't know which ones.

Data loss is weighted by P2, am I right?
Please, correct the name of the issue (by deleting "from user's view") if You
agree with me.
Comment 18 tuharsky 2006-02-06 13:38:50 UTC

"User data is corrupted in an easy-to-encounter way;
e.g. saving a document corrupts the resulting file and renders it unusable"
Comment 19 tuharsky 2006-03-08 09:04:03 UTC

"User data is corrupted in an easy-to-encounter way;
e.g. saving a document corrupts the resulting file and renders it unusable

Issues with this priority must be fixed before the target release (see Target
milestone), which usually is the next major release, and should be dealt with as
soon as possible. Not fixing them for the target release is not acceptable."

I can accept, that there is lack of developers. However I don't think that
hiding the problems and turning face away from them really helps the OOo, nor
does it help users. Is it so hard to face the problems and just say "Well, there
are too many issues in OOo and we don't have enough time/people/fun/whatever
fixing them"??

Data lost (P2) issue from 10/2004 that lasts year and half (one major and 4
minor versions -still there in 2.0.2rc5) would look too bad, but if we mark it
P3 and "enhancement", looks better on paper/web?

Please, anybody capable, at least restore the proper attributes of this bug.
Comment 20 michael.ruess 2006-03-08 09:13:25 UTC
Removing mru from cc.
Comment 21 Martin Hollmichel 2006-09-28 12:52:33 UTC
if this is confirmed as data loss, we set target for next release.
Comment 22 niklas.nebel 2008-01-14 16:57:43 UTC
Fixed on CWS "osfme01". If there's any text cell in a column, the column's type
is now text.
Comment 23 niklas.nebel 2008-01-14 17:01:32 UTC
changing type
Comment 24 niklas.nebel 2008-01-14 18:21:51 UTC
back to QA for verification (who takes care of mail merge issues?)
Comment 25 michael.ruess 2008-01-16 14:34:16 UTC
MRU->HI: pls verify this in the CWS.
Comment 26 h.ilter 2008-01-16 19:10:36 UTC
Verified with cws osfme01 = looks good for me.
Comment 27 h.ilter 2008-01-22 12:48:54 UTC
Same in c21v001
Comment 28 niklas.nebel 2008-05-13 15:44:39 UTC
reopening for CWS fmebugs04
Comment 29 niklas.nebel 2008-05-13 15:45:26 UTC
Fixed also on CWS fmebugs04.
Comment 30 h.ilter 2008-05-20 15:45:40 UTC
Still ok in fmebugs04
Comment 31 h.ilter 2008-09-15 20:42:23 UTC
*** Issue 93858 has been marked as a duplicate of this issue. ***
Comment 32 Marcus 2008-10-07 14:57:33 UTC
set the target to 2.4.2
Comment 33 h.ilter 2008-10-08 08:34:27 UTC
Reopened and reassigned due to changed target
Comment 34 h.ilter 2008-10-08 08:35:06 UTC
Comment 35 Oliver Specht 2008-10-08 09:09:22 UTC
reassigned to nn
Comment 36 stefan.baltzer 2008-10-16 17:34:52 UTC
Correcting target. 
This one was fixed for OOo 3.0 and will NOT be fixed for OOo 2.4.2.
Reassigned to HI.
Comment 37 stefan.baltzer 2008-10-16 17:36:14 UTC
Set to FIXED again.
Comment 38 stefan.baltzer 2008-10-16 17:37:58 UTC
Set to Verified (in CWS) again.
SBA->HI: Please re-verify in OOo 3.0 Final and close, thank you.
Comment 39 h.ilter 2008-10-17 13:15:47 UTC
Comment 40 larryk2lt 2010-06-22 01:38:39 UTC
missing data in database after creating from spreadsheet.  Also format of
columns not carried through either.  OO 3.2.1 Windows XP SP3.  Issue 35178 is
good description of problem.  June 21, 2010.
Comment 41 tuharsky 2010-06-22 08:08:59 UTC
Comment from larryk2lt might indicate a regression. Therefore I'm reopening the
issue in order to get things investigated and recorded. Or is it necessarry to
open a new issue for regression?

larryk2lt: Can You replicate the problem and confirm that it is exactly this one?
Comment 42 h.ilter 2010-06-22 10:50:12 UTC
Not a regression because it's already broken in OOo3.2
But it will be fixed to OOo 3.3
Please try the developer snapshot m83
Comment 43 h.ilter 2010-06-22 10:51:03 UTC