Nigel Vickers | 17 Dec 08:01 2011

Missing Gridfs collection fs.file

Good Morning,
I was introduced to mongoDB while working with go and experimenting with the superb driver mgo from Gustavo Niemeyer. I wish to use gridfs. when loading a file using mongofiles the "put" returns "assertion: 9008 filemd5 failed". The fs.chunks are being correctly inserted into to the relevant db but fs.files is not being created. The reason for the md5 failure is not transparent. I think the missing fs.files is part of the problem. I have tried to create the collection by hand without success. I have seen this error in the lists but in conjunction with a sharding problem. I am not using sharding at the moment.  I would be very grateful if someone could point me in the right direction.

Before:
sysop <at> rhedile-dev:~/falkenstein/html5$ uname -a
Linux rhedile-dev 2.6.32-33-server #72-Ubuntu SMP Fri Jul 29 21:21:55 UTC 2011 x86_64 GNU/Linux
sysop <at> rhedile-dev:~/falkenstein/html5$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=10.04
DISTRIB_CODENAME=lucid
DISTRIB_DESCRIPTION="Ubuntu 10.04.3 LTS"

mongo
> db.version()
2.0.1

> db.printShardingStatus()
printShardingStatus: not a shard db!

> show collections
articledata
atemplates
formdata
images
js
pages
ptemplates
sitecss
system.indexes
system.users
templates

Console load: 
sysop <at> rhedile-dev:~/falkenstein/html5$  mongofiles -vv -u "xxxxdba" -p "xxxxxx" -d falkenstein  put 1mbtest
Sat Dec 17 07:09:35 creating new connection to:127.0.0.1
Sat Dec 17 07:09:35 BackgroundJob starting: ConnectBG
Sat Dec 17 07:09:35 connected connection!
connected to: 127.0.0.1
assertion: 9008 filemd5 failed

mongo.log
Sat Dec 17 07:09:35 [conn5201] New namespace: falkenstein.fs.chunks
Sat Dec 17 07:09:35 [conn5201] reset extent was:falkenstein.fs.chunks now:falkenstein.fs.chunks
adding _id index for collection falkenstein.fs.chunks
Sat Dec 17 07:09:35 [conn5201] build index falkenstein.fs.chunks { _id: 1 }
mem info: before index start vsize: 1192 resident: 172 mapped: 448
Sat Dec 17 07:09:35 [conn5201] external sort root: /var/lib/mongodb/_tmp/esort.1324102175.469017013/
mem info: before final sort vsize: 1192 resident: 172 mapped: 448
mem info: after final sort vsize: 1192 resident: 172 mapped: 448
Sat Dec 17 07:09:35 [conn5201]   external sort used : 0 files  in 0 secs
Sat Dec 17 07:09:35 [conn5201] New namespace: falkenstein.fs.chunks.$_id_
Sat Dec 17 07:09:35 [conn5201] reset extent was:falkenstein.fs.chunks.$_id_ now:falkenstein.fs.chunks.$_id_
         done building bottom layer, going to commit
Sat Dec 17 07:09:35 [conn5201]   fastBuildIndex dupsToDrop:0
Sat Dec 17 07:09:35 [conn5201] build index done 0 records 0.001 secs
Sat Dec 17 07:09:35 [conn5201] insert falkenstein.fs.chunks 2ms
Sat Dec 17 07:09:35 [conn5201] insert falkenstein.fs.chunks 0ms
Sat Dec 17 07:09:35 [conn5201] insert falkenstein.fs.chunks 0ms
Sat Dec 17 07:09:35 [conn5201] insert falkenstein.fs.chunks 0ms
Sat Dec 17 07:09:35 [conn5201] insert falkenstein.fs.chunks 0ms
Sat Dec 17 07:09:35 [conn5201] runQuery called falkenstein.$cmd { filemd5: ObjectId('4eec321fcfb7e580e919cf5e'), root: "fs" }
Sat Dec 17 07:09:35 [conn5201] run command falkenstein.$cmd { filemd5: ObjectId('4eec321fcfb7e580e919cf5e'), root: "fs" }
Sat Dec 17 07:09:35 [conn5201] warning: best guess query plan requested, but scan and order are required for all plans  query: { files_id: ObjectId('4eec321fcfb7e580e919cf5e') } order: { files_id: 1, n: 1 } choices: { $natural: 1 } 
Sat Dec 17 07:09:35 [conn5201] command falkenstein.$cmd command: { filemd5: ObjectId('4eec321fcfb7e580e919cf5e'), root: "fs" } ntoreturn:1 reslen:91 0ms
Sat Dec 17 07:09:35 [conn5201] Socket recv() conn closed? 127.0.0.1:46320
Sat Dec 17 07:09:35 [conn5201] SocketException: remote: 127.0.0.1:46320 error: 9001 socket exception [0] server [127.0.0.1:46320
Sat Dec 17 07:09:35 [conn5201] end connection 127.0.0.1:46320


After:
show collections
articledata
atemplates
formdata
fs.chunks
images
js
pages
ptemplates
sitecss
system.indexes
system.users
templates
> db.fs.chunks.find({},{files_id:1})
{ "_id" : ObjectId("4eec321f7881f70c9bee7cfb"), "files_id" : ObjectId("4eec321fcfb7e580e919cf5e") }
{ "_id" : ObjectId("4eec321f7881f70c9bee7cfc"), "files_id" : ObjectId("4eec321fcfb7e580e919cf5e") }
{ "_id" : ObjectId("4eec321f7881f70c9bee7cfd"), "files_id" : ObjectId("4eec321fcfb7e580e919cf5e") }
{ "_id" : ObjectId("4eec321f7881f70c9bee7cfe"), "files_id" : ObjectId("4eec321fcfb7e580e919cf5e") }
{ "_id" : ObjectId("4eec321f7881f70c9bee7cff"), "files_id" : ObjectId("4eec321fcfb7e580e919cf5e") }


Regards,
Nigel Vickers

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to mongodb-user+unsubscribe <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
Eliot Horowitz | 17 Dec 08:06 2011

Re: Missing Gridfs collection fs.file

There needs to be an index on { files_id : 1 , n : 1 } on the chunk collection.
Looks like its missing.

On Sat, Dec 17, 2011 at 2:01 AM, Nigel Vickers <rhedile@...> wrote:
> Good Morning,
> I was introduced to mongoDB while working with go and experimenting with the
> superb driver mgo from Gustavo Niemeyer. I wish to use gridfs. when loading
> a file using mongofiles the "put" returns "assertion: 9008 filemd5 failed".
> The fs.chunks are being correctly inserted into to the relevant db but
> fs.files is not being created. The reason for the md5 failure is not
> transparent. I think the missing fs.files is part of the problem. I have
> tried to create the collection by hand without success. I have seen this
> error in the lists but in conjunction with a sharding problem. I am not
> using sharding at the moment.  I would be very grateful if someone could
> point me in the right direction.
>
> Before:
> sysop <at> rhedile-dev:~/falkenstein/html5$ uname -a
> Linux rhedile-dev 2.6.32-33-server #72-Ubuntu SMP Fri Jul 29 21:21:55 UTC
> 2011 x86_64 GNU/Linux
> sysop <at> rhedile-dev:~/falkenstein/html5$ cat /etc/lsb-release
> DISTRIB_ID=Ubuntu
> DISTRIB_RELEASE=10.04
> DISTRIB_CODENAME=lucid
> DISTRIB_DESCRIPTION="Ubuntu 10.04.3 LTS"
>
> mongo
>> db.version()
> 2.0.1
>
>> db.printShardingStatus()
> printShardingStatus: not a shard db!
>>
>
>> show collections
> articledata
> atemplates
> formdata
> images
> js
> pages
> ptemplates
> sitecss
> system.indexes
> system.users
> templates
>>
>
> Console load:
> sysop <at> rhedile-dev:~/falkenstein/html5$  mongofiles -vv -u "xxxxdba" -p
> "xxxxxx" -d falkenstein  put 1mbtest
> Sat Dec 17 07:09:35 creating new connection to:127.0.0.1
> Sat Dec 17 07:09:35 BackgroundJob starting: ConnectBG
> Sat Dec 17 07:09:35 connected connection!
> connected to: 127.0.0.1
> assertion: 9008 filemd5 failed
>
> mongo.log
> Sat Dec 17 07:09:35 [conn5201] New namespace: falkenstein.fs.chunks
> Sat Dec 17 07:09:35 [conn5201] reset extent was:falkenstein.fs.chunks
> now:falkenstein.fs.chunks
> adding _id index for collection falkenstein.fs.chunks
> Sat Dec 17 07:09:35 [conn5201] build index falkenstein.fs.chunks { _id: 1 }
> mem info: before index start vsize: 1192 resident: 172 mapped: 448
> Sat Dec 17 07:09:35 [conn5201] external sort root:
> /var/lib/mongodb/_tmp/esort.1324102175.469017013/
> mem info: before final sort vsize: 1192 resident: 172 mapped: 448
> mem info: after final sort vsize: 1192 resident: 172 mapped: 448
> Sat Dec 17 07:09:35 [conn5201]   external sort used : 0 files  in 0 secs
> Sat Dec 17 07:09:35 [conn5201] New namespace: falkenstein.fs.chunks.$_id_
> Sat Dec 17 07:09:35 [conn5201] reset extent was:falkenstein.fs.chunks.$_id_
> now:falkenstein.fs.chunks.$_id_
>          done building bottom layer, going to commit
> Sat Dec 17 07:09:35 [conn5201]   fastBuildIndex dupsToDrop:0
> Sat Dec 17 07:09:35 [conn5201] build index done 0 records 0.001 secs
> Sat Dec 17 07:09:35 [conn5201] insert falkenstein.fs.chunks 2ms
> Sat Dec 17 07:09:35 [conn5201] insert falkenstein.fs.chunks 0ms
> Sat Dec 17 07:09:35 [conn5201] insert falkenstein.fs.chunks 0ms
> Sat Dec 17 07:09:35 [conn5201] insert falkenstein.fs.chunks 0ms
> Sat Dec 17 07:09:35 [conn5201] insert falkenstein.fs.chunks 0ms
> Sat Dec 17 07:09:35 [conn5201] runQuery called falkenstein.$cmd { filemd5:
> ObjectId('4eec321fcfb7e580e919cf5e'), root: "fs" }
> Sat Dec 17 07:09:35 [conn5201] run command falkenstein.$cmd { filemd5:
> ObjectId('4eec321fcfb7e580e919cf5e'), root: "fs" }
> Sat Dec 17 07:09:35 [conn5201] warning: best guess query plan requested, but
> scan and order are required for all plans  query: { files_id:
> ObjectId('4eec321fcfb7e580e919cf5e') } order: { files_id: 1, n: 1 } choices:
> { $natural: 1 }
> Sat Dec 17 07:09:35 [conn5201] command falkenstein.$cmd command: { filemd5:
> ObjectId('4eec321fcfb7e580e919cf5e'), root: "fs" } ntoreturn:1 reslen:91 0ms
> Sat Dec 17 07:09:35 [conn5201] Socket recv() conn closed? 127.0.0.1:46320
> Sat Dec 17 07:09:35 [conn5201] SocketException: remote: 127.0.0.1:46320
> error: 9001 socket exception [0] server [127.0.0.1:46320]
> Sat Dec 17 07:09:35 [conn5201] end connection 127.0.0.1:46320
>
>
> After:
> show collections
> articledata
> atemplates
> formdata
> fs.chunks
> images
> js
> pages
> ptemplates
> sitecss
> system.indexes
> system.users
> templates
>> db.fs.chunks.find({},{files_id:1})
> { "_id" : ObjectId("4eec321f7881f70c9bee7cfb"), "files_id" :
> ObjectId("4eec321fcfb7e580e919cf5e") }
> { "_id" : ObjectId("4eec321f7881f70c9bee7cfc"), "files_id" :
> ObjectId("4eec321fcfb7e580e919cf5e") }
> { "_id" : ObjectId("4eec321f7881f70c9bee7cfd"), "files_id" :
> ObjectId("4eec321fcfb7e580e919cf5e") }
> { "_id" : ObjectId("4eec321f7881f70c9bee7cfe"), "files_id" :
> ObjectId("4eec321fcfb7e580e919cf5e") }
> { "_id" : ObjectId("4eec321f7881f70c9bee7cff"), "files_id" :
> ObjectId("4eec321fcfb7e580e919cf5e") }
>>
>
>
> Regards,
> Nigel Vickers
>
> --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To post to this group, send email to mongodb-user@...
> To unsubscribe from this group, send email to
> mongodb-user+unsubscribe@...
> For more options, visit this group at
> http://groups.google.com/group/mongodb-user?hl=en.

--

-- 
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongodb-user@...
To unsubscribe from this group, send email to mongodb-user+unsubscribe <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

Nigel Vickers | 17 Dec 08:50 2011

Re: Missing Gridfs collection fs.file

Hello Eliot,
That was it thanks for the help.
On Dec 17, 8:06 am, Eliot Horowitz <el...@...> wrote:
> There needs to be an index on { files_id : 1 , n : 1 } on the chunk collection.
> Looks like its missing.
>
From the documentation:

db.fs.chunks.ensureIndex({files_id:1, n:1},{unique:true})

works wonders.

 However, although this index is required it apparently is not being
created for a new db when gridfs is being used for the first time. Is
this intended?

Regards
Nigel Vickers

--

-- 
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongodb-user@...
To unsubscribe from this group, send email to mongodb-user+unsubscribe <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

Eliot Horowitz | 17 Dec 16:02 2011

Re: Re: Missing Gridfs collection fs.file

The gridfs driver is supposed to add the index.

On Sat, Dec 17, 2011 at 2:50 AM, Nigel Vickers <rhedile@...> wrote:
> Hello Eliot,
> That was it thanks for the help.
> On Dec 17, 8:06 am, Eliot Horowitz <el...@...> wrote:
>> There needs to be an index on { files_id : 1 , n : 1 } on the chunk collection.
>> Looks like its missing.
>>
> From the documentation:
>
> db.fs.chunks.ensureIndex({files_id:1, n:1},{unique:true})
>
> works wonders.
>
>  However, although this index is required it apparently is not being
> created for a new db when gridfs is being used for the first time. Is
> this intended?
>
> Regards
> Nigel Vickers
>
> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongodb-user@...
> To unsubscribe from this group, send email to mongodb-user+unsubscribe <at> googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>

--

-- 
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongodb-user@...
To unsubscribe from this group, send email to mongodb-user+unsubscribe <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

Gustavo Niemeyer | 21 Dec 18:01 2011
Picon

Re: Re: Missing Gridfs collection fs.file

Hi again Nigel,

>> There needs to be an index on { files_id : 1 , n : 1 } on the chunk collection.
>> Looks like its missing.
>
> From the documentation:
>
> db.fs.chunks.ensureIndex({files_id:1, n:1},{unique:true})

Sorry, I've missed your message from the mongodb-users list.

How did you add the files to the GridFS?

mgo will ensure that the index on files_id and n exists every time you
try to add a file to the GridFS while using the driver.

--

-- 
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/plus
http://niemeyer.net/twitter
http://niemeyer.net/blog

-- I'm not absolutely sure of anything.


Gmane