LMDB-Patches and SQLightning-Issues

Discussion:

Johannes Munk (P5)

2015-04-28 16:41:34 UTC

Dear Howard and OpenLDAP-team!

First of all: thanks a lot for the great LMDB and all the effort you
have put into it.

In the last couple of weeks we have been trying to get LMDB/SQLightning
to work as backend for our semantic network software.

On that road we made LMDB compatible with Win8.1/MSVC2013 (see patches
here <https://gitlab.com/lukas-w/lmdb/commits/p5>), fixing scrambled
filenames
<https://gitlab.com/lukas-w/lmdb/commit/e10837272788f87ce43049bbbd3c9bcc0071fa68>
and made SQLightning work with files greater than 4 GB
<https://gitlab.com/lukas-w/sqlightning/commit/e52a9949fd5c30b7ac45b0909259c793e4c1ee4f>.

But after some simple tests that worked, we ran into some issues, which
we can't get a handle on. We have reported most of our issues here:
https://gitlab.com/mdb/sqlightning/issues

It would be great, if you could take a peek!

Thx a lot in advance,

Johannes

Howard Chu

2015-04-29 02:24:03 UTC

Permalink

Content preview: Johannes Munk (P5) wrote: > Dear Howard and OpenLDAP-team!

Post by Johannes Munk (P5)
First of all: thanks a lot for the great LMDB and all the effort you

have put into it. > > In the last couple of weeks we have been trying to

get LMDB/SQLightning > to work as backend for our semantic network software.

Post by Johannes Munk (P5)
On that road we made LMDB compatible with Win8.1/MSVC2013 (see patches

here <https://gitlab.com/lukas-w/lmdb/commits/p5>), fixing scrambled >

filenames > <https://gitlab.com/lukas-w/lmdb/commit/e10837272788f87ce43049bbbd3c9bcc0071fa68>

and made SQLightning work with files greater than 4 GB > <https://gitlab.com/lukas-w/sqlightning/commit/e52a9949fd5c30b7ac45b0909259c793e4c1ee4f>.

Post by Johannes Munk (P5)
But after some simple tests that worked, we ran into some issues, which

we can't get a handle on. We have reported most of our issues here: > https://gitlab.com/mdb/sqlightning/issues

Post by Johannes Munk (P5)
It would be great, if you could take a peek! [...]

Content analysis details: (-4.2 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/, medium
trust
[69.43.206.106 listed in list.dnswl.org]
-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
[score: 0.0000]

Dear Howard and OpenLDAP-team!
First of all: thanks a lot for the great LMDB and all the effort you
have put into it.
In the last couple of weeks we have been trying to get LMDB/SQLightning
to work as backend for our semantic network software.
On that road we made LMDB compatible with Win8.1/MSVC2013 (see patches
here <https://gitlab.com/lukas-w/lmdb/commits/p5>), fixing scrambled
filenames
<https://gitlab.com/lukas-w/lmdb/commit/e10837272788f87ce43049bbbd3c9bcc0071fa68>
and made SQLightning work with files greater than 4 GB
<https://gitlab.com/lukas-w/sqlightning/commit/e52a9949fd5c30b7ac45b0909259c793e4c1ee4f>.
But after some simple tests that worked, we ran into some issues, which
https://gitlab.com/mdb/sqlightning/issues
It would be great, if you could take a peek!

Hi. Will take a look when time is available. Just to note, importing the
LMDB repos from gitorious to gitlab was unsuccessful/incomplete and we
never found any diagnostics on gitlab to indicate why. As such we have
given up on using both gitorious and gitlab. Ongoing work with LMDB
subprojects will be on github instead.

https://github.com/LMDB

Development on LMDB itself will continue to occur in the main OpenLDAP
git repos, with read-only mirroring to github.

--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/

Johannes Munk (P5)

2015-05-16 00:06:21 UTC

Permalink

Dear Howard,

thank you, it would indeed be great, if you could take a look.

Since my last mail we were able to track down two additional problems.
The first one really stops our show - I can't work around it.

1) Some SQL queries abort execution instead of returning values. It
occurs for queries/parameters without obvious connection. In the issue
on gitlab <https://gitlab.com/mdb/sqlightning/issues/5> we found a
simple set of statements reproducing the problem.

2) Indices get corrupted <https://gitlab.com/mdb/sqlightning/issues/4>
if keys longer than 16kB are inserted. We simply avoided keys that long
for now..

We could start using sqlightning and thus have real multithreading on
our database, if you could fix the issue with the aborted statements.

Let me know, if I can do anything to support you in doing so.

Concerning repositories: Shall we move the issues to some other place?

Much appreciated,

Johannes Munk

Post by Howard Chu

Post by Johannes Munk (P5)
Dear Howard and OpenLDAP-team!
First of all: thanks a lot for the great LMDB and all the effort you
have put into it.
In the last couple of weeks we have been trying to get LMDB/SQLightning
to work as backend for our semantic network software.
On that road we made LMDB compatible with Win8.1/MSVC2013 (see patches
here <https://gitlab.com/lukas-w/lmdb/commits/p5>), fixing scrambled
filenames
<https://gitlab.com/lukas-w/lmdb/commit/e10837272788f87ce43049bbbd3c9bcc0071fa68>
and made SQLightning work with files greater than 4 GB
<https://gitlab.com/lukas-w/sqlightning/commit/e52a9949fd5c30b7ac45b0909259c793e4c1ee4f>.
But after some simple tests that worked, we ran into some issues, which
https://gitlab.com/mdb/sqlightning/issues
It would be great, if you could take a peek!

Hi. Will take a look when time is available. Just to note, importing
the LMDB repos from gitorious to gitlab was unsuccessful/incomplete
and we never found any diagnostics on gitlab to indicate why. As such
we have given up on using both gitorious and gitlab. Ongoing work with
LMDB subprojects will be on github instead.
https://github.com/LMDB
Development on LMDB itself will continue to occur in the main OpenLDAP
git repos, with read-only mirroring to github.

Howard Chu

2015-05-16 12:00:53 UTC

Permalink

Content preview: Johannes Munk (P5) wrote: > Concerning repositories: Shall
we move the issues to some other place? Yes, please switch to github. I will
be deleting the gitlab repos and closing that account. [...]

Content analysis details: (-4.2 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/, medium
trust
[69.43.206.106 listed in list.dnswl.org]
-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
[score: 0.0000]

Post by Johannes Munk (P5)
Concerning repositories: Shall we move the issues to some other place?

Yes, please switch to github. I will be deleting the gitlab repos and closing
that account.

https://github.com/LMDB

gitlab is too unreliable and provides no diagnostic messages when it fails.

Post by Johannes Munk (P5)
Much appreciated,
Johannes Munk

Post by Howard Chu

Hi. Will take a look when time is available. Just to note, importing the
LMDB repos from gitorious to gitlab was unsuccessful/incomplete and we never
found any diagnostics on gitlab to indicate why. As such we have given up on
using both gitorious and gitlab. Ongoing work with LMDB subprojects will be
on github instead.
https://github.com/LMDB
Development on LMDB itself will continue to occur in the main OpenLDAP git
repos, with read-only mirroring to github.

--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/

Johannes Munk (P5)

2015-05-17 12:55:19 UTC

Permalink

Content preview: Hello Howard, as requested, I copied the issues to your github
repo: https://github.com/LMDB/sqlightning/issues Number 5 is the most critical
one for us https://github.com/LMDB/sqlightning/issues/5 followed by number
2 https://github.com/LMDB/sqlightning/issues/2 [...]

Content analysis details: (-2.6 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low
trust
[85.119.153.27 listed in list.dnswl.org]
-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
[score: 0.0000]

Hello Howard,

as requested, I copied the issues to your github repo:
https://github.com/LMDB/sqlightning/issues

Number 5 is the most critical one for us
https://github.com/LMDB/sqlightning/issues/5
followed by number 2
https://github.com/LMDB/sqlightning/issues/2

Again: Let me know, if I can do anything to help you.

Johannes

Post by Howard Chu

Post by Johannes Munk (P5)
Concerning repositories: Shall we move the issues to some other place?

Yes, please switch to github. I will be deleting the gitlab repos and
closing that account.
https://github.com/LMDB
gitlab is too unreliable and provides no diagnostic messages when it fails.

Post by Johannes Munk (P5)
Much appreciated,
Johannes Munk

Post by Howard Chu

Post by Johannes Munk (P5)
Dear Howard and OpenLDAP-team!
First of all: thanks a lot for the great LMDB and all the effort you
have put into it.
In the last couple of weeks we have been trying to get
LMDB/SQLightning
to work as backend for our semantic network software.
On that road we made LMDB compatible with Win8.1/MSVC2013 (see patches
here <https://gitlab.com/lukas-w/lmdb/commits/p5>), fixing scrambled
filenames
<https://gitlab.com/lukas-w/lmdb/commit/e10837272788f87ce43049bbbd3c9bcc0071fa68>
and made SQLightning work with files greater than 4 GB
<https://gitlab.com/lukas-w/sqlightning/commit/e52a9949fd5c30b7ac45b0909259c793e4c1ee4f>.
But after some simple tests that worked, we ran into some issues, which
https://gitlab.com/mdb/sqlightning/issues
It would be great, if you could take a peek!

Hi. Will take a look when time is available. Just to note, importing the
LMDB repos from gitorious to gitlab was unsuccessful/incomplete and we never
found any diagnostics on gitlab to indicate why. As such we have given up on
using both gitorious and gitlab. Ongoing work with LMDB subprojects will be
on github instead.
https://github.com/LMDB
Development on LMDB itself will continue to occur in the main OpenLDAP git
repos, with read-only mirroring to github.