headers
Svip | 1 Apr 2012 14:53
Picon

Re: Time to redirect to https by default?

On 1 April 2012 13:59, David Gerard <dgerard <at> gmail.com> wrote:

> On 1 April 2012 12:23, Svip <svippy <at> gmail.com> wrote:
>
>> On 1 April 2012 12:06, David Gerard <dgerard <at> gmail.com> wrote:
>>
>>> http://www.bbc.co.uk/news/uk-politics-17576745
>>
>> Also, this article was written on 1 April and is far beyond any
>> monitoring scheme ever suggested in the Western World.  And I am sure
>> we would have heard about it being mentioned up until this point, if
>> it was real.
>
> It would be nice, but if it's a prank then (a) lots of other
> newspapers are in on it (b) ORG flagged the programme described
> several weeks in advance:
>
> http://wiki.openrightsgroup.org/wiki/Communications_Capabilities_Development_Programme
> http://www.openrightsgroup.org/issues/ccdp
>
> So no, it's in no way a joke. This is absolutely real.

Still *kind of* a joke.

>> So I would take that article with a grain of salt.  Particularly the
>> statement about 'real time'.  That's not even feasible.
>
> That a desired monitoring regime would require a violation of physics
> has *never* stopped a legislative push for such.
(Continue reading)

Permalink | Reply |
headers
Piotr Jagielski | 1 Apr 2012 16:04
Picon
Favicon

correct way to import SQL dumps into MySQL database in terms of character encoding

Hello,

I'm trying to import categorylinks.sql dump into my MySQL database. I'm 
able to import it and query for articles in specific categories as long 
the category name contains only English-language characters. I don't get 
any results if I try to query for non-English category name. My 
understanding is that the dump is in UTF-8 format so I tried the following:

create the database using the following command:
CREATE DATABASE wiki CHARACTER SET utf8 COLLATE utf8_general_ci;

import the dump using the following command:
mysql --user root --password=root wiki < 
C:\Path\plwiki-20111227-categorylinks.sql --default-character-set=utf8

set my data source URL to the following in my Java code:
jdbc:mysql://localhost/plwiki?useUnicode=true&characterEncoding=UTF-8

It still doesn't work. What am I missing? Are there any instructions on 
how to correctly import the dump anywhere?

Thanks,
Piotr
Permalink | Reply |
headers
Svip | 1 Apr 2012 16:31
Picon

Re: correct way to import SQL dumps into MySQL database in terms of character encoding

On 1 April 2012 16:04, Piotr Jagielski <piotr.jagielski <at> op.pl> wrote:

> mysql --user root --password=root wiki <
> C:\Path\plwiki-20111227-categorylinks.sql --default-character-set=utf8

It's -p, not --password=root and it will prompt you for the password.
Permalink | Reply |
headers
Piotr Jagielski | 1 Apr 2012 17:05
Picon
Favicon

Re: correct way to import SQL dumps into MySQL database in terms of character encoding

These options should be equivalent. It does load the data using the 
below command. It just incorrectly handles non-English characters.

Regards,
Piotr

On 2012-04-01 16:31, Svip wrote:
> On 1 April 2012 16:04, Piotr Jagielski<piotr.jagielski <at> op.pl>  wrote:
>
>> mysql --user root --password=root wiki<
>> C:\Path\plwiki-20111227-categorylinks.sql --default-character-set=utf8
> It's -p, not --password=root and it will prompt you for the password.
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l <at> lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
Permalink | Reply |
headers
Platonides | 1 Apr 2012 17:30
Picon

Re: correct way to import SQL dumps into MySQL database in terms of character encoding

On 01/04/12 17:05, Piotr Jagielski wrote:
> These options should be equivalent. It does load the data using the
> below command. It just incorrectly handles non-English characters.
> 
> Regards,
> Piotr

Do you have $wgDBmysql5 set in your LocalSettings.php?
Permalink | Reply |
headers
Piotr Jagielski | 1 Apr 2012 17:37
Picon
Favicon

Re: correct way to import SQL dumps into MySQL database in terms of character encoding

I don't have MediaWiki installed. I'm just trying to import the dump 
into a standalone database so I can do some batch processing on the data.

Regards,
Piotr

On 2012-04-01 17:30, Platonides wrote:
> On 01/04/12 17:05, Piotr Jagielski wrote:
>> These options should be equivalent. It does load the data using the
>> below command. It just incorrectly handles non-English characters.
>>
>> Regards,
>> Piotr
> Do you have $wgDBmysql5 set in your LocalSettings.php?
>
>
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l <at> lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
Permalink | Reply |
headers
Platonides | 1 Apr 2012 22:30
Picon

Re: correct way to import SQL dumps into MySQL database in terms of character encoding

On 01/04/12 17:37, Piotr Jagielski wrote:
> I don't have MediaWiki installed. I'm just trying to import the dump
> into a standalone database so I can do some batch processing on the data.
> 
> Regards,
> Piotr

It inserts the data fine for me. I suspect your java code is failing to
appropiately read them. Try reading the table with a different tool,
such as phpMyAdmin.

> mysql> select * from categorylinks limit 20;
> +---------+---------------------------------------+-------------------------------------+---------------------+-------------------+--------------+---------+
> | cl_from | cl_to                                 | cl_sortkey                          | cl_timestamp        | cl_sortkey_prefix | cl_collation | cl_type |
> +---------+---------------------------------------+-------------------------------------+---------------------+-------------------+--------------+---------+
> |       0 | Ekspresowe_kasowanko                  | Golembiovski Andzey                 | 2009-07-09 21:01:30 |                   |              | page    |
> |       2 | Języki_skryptowe                      | AWK
> AWK                             | 2011-01-18 01:11:23 | Awk               | uppercase    | page    |
> |       4 | Specjalności_lekarskie                | ALERGOLOGIA                         | 2008-04-25 10:31:22 |                   | uppercase    | page    |
> |       6 | Formaty_plików_komputerowych          | ASCII                               | 2011-09-23 11:01:05 |                   | uppercase    | page    |
> |       6 | Kodowania_znaków                      | ASCII                               | 2011-09-23 11:01:05 |                   | uppercase    | page    |
> |       7 | Artykuły_na_medal                     | ATOM                                | 2010-12-01 16:40:37 |                   | uppercase    | page    |
> |       7 | Artykuły_wymagające_dopracowania      | ATOM                                | 2011-08-16 15:53:43 |                   | uppercase    | page    |
> |       7 | Atomy                                 |  
> ATOM                              | 2011-08-09 00:56:39 |                   | uppercase    | page    |
> |       8 | Logika_matematyczna                   | AKSJOMAT                            | 2007-11-10 08:18:06 |                   | uppercase    | page    |
> |      10 | Arytmetyka                            |  
> ARYTMETYKA                        | 2011-10-17 02:36:39 |                   | uppercase    | page    |
> |      11 | Artykuły_pod_opieką_Projektu_Chemia   | AMINOKWASY                          | 2011-08-19 02:48:21 |                   | uppercase    | page    |
> |      12 | Alkeny                                | *
(Continue reading)

Permalink | Reply |
headers
Marcin Cieslak | 1 Apr 2012 19:50
Gravatar

Re: correct way to import SQL dumps into MySQL database in terms of character encoding

>> Piotr Jagielski <piotr.jagielski <at> op.pl> wrote:
> Hello,
>
> set my data source URL to the following in my Java code:
> jdbc:mysql://localhost/plwiki?useUnicode=true&characterEncoding=UTF-8

Please note you have "plwiki" here and you imported into "wiki".
Assuming your .my.cnf is not making things difficult I ran a small
Jython script to test:

$ jython
Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06) 
[OpenJDK 64-Bit Server VM (Sun Microsystems Inc.)] on java1.6.0
Type "help", "copyright", "credits" or "license" for more information.
>>> from com.ziclix.python.sql import zxJDBC
>>> d, u, p, v = "jdbc:mysql://localhost/wiki", "root", None, "org.gjt.mm.mysql.Driver"
>>> db = zxJDBC.connect(d, u, p, v, CHARSET="utf8")
>>> c=db.cursor()
>>> c.execute("select cl_from, cl_to from categorylinks where cl_from=61 limit 10")
>>> c.fetchone()
(61, array('b', [65, 110, 100, 111, 114, 97]))
>>> (a,b) = c.fetchone()
>>> print b
array('b', [67, 122, -59, -126, 111, 110, 107, 111, 119, 105, 101, 95, 79, 114, 103, 97, 110, 105, 122, 97, 99,
106, 105, 95, 78, 97, 114, 111, 100, -61, -77, 119, 95, 90, 106, 101, 100, 110, 111, 99, 122, 111, 110, 121, 99, 104])
>>> for x in b:
...     try:
...         print chr(x),
...     except ValueError:
...         print "%02x" % x,
(Continue reading)

Permalink | Reply |
headers
Piotr Jagielski | 1 Apr 2012 20:32
Picon
Favicon

Re: correct way to import SQL dumps into MySQL database in terms of character encoding

Sorry, I made a mistake in the e-mail. I had the database set to the 
same name in both places.

My problem is actually opposite because I don't get any result where I 
use UTF-8 string as an input in the query. But I verified that I don't 
get correct results where using the query you provided neither. The link 
with the MySQL bug report might be helpful in resolving the problem so 
thanks for providing it.

Piotr

On 2012-04-01 19:50, Marcin Cieslak wrote:
>>> Piotr Jagielski<piotr.jagielski <at> op.pl>  wrote:
>> Hello,
>>
>> set my data source URL to the following in my Java code:
>> jdbc:mysql://localhost/plwiki?useUnicode=true&characterEncoding=UTF-8
> Please note you have "plwiki" here and you imported into "wiki".
> Assuming your .my.cnf is not making things difficult I ran a small
> Jython script to test:
>
> $ jython
> Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06)
> [OpenJDK 64-Bit Server VM (Sun Microsystems Inc.)] on java1.6.0
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from com.ziclix.python.sql import zxJDBC
>>>> d, u, p, v = "jdbc:mysql://localhost/wiki", "root", None, "org.gjt.mm.mysql.Driver"
>>>> db = zxJDBC.connect(d, u, p, v, CHARSET="utf8")
>>>> c=db.cursor()
>>>> c.execute("select cl_from, cl_to from categorylinks where cl_from=61 limit 10")
(Continue reading)

Permalink | Reply |
headers
Platonides | 1 Apr 2012 17:28
Picon

Re: Time to redirect to https by default?

On 1 April 2012 14:53, Svip wrote:
> On 1 April 2012 13:59, David Gerard <dgerard <at> gmail.com> wrote:
>> On 1 April 2012 12:23, Svip <svippy <at> gmail.com> wrote:
>>> So I would take that article with a grain of salt.  Particularly the
>>> statement about 'real time'.  That's not even feasible.
>>
>> That a desired monitoring regime would require a violation of physics
>> has *never* stopped a legislative push for such.
> 
> But it has always stopped it from being implemented or executed in
> practice.  While the development is terrifying, it is also important
> to note the lack of actual consequences it will have.  Other than
> being a huge embarrassment.

I don't see why it *couldn't* be implemented.
Note that the real time statement is no different on how they can snoop
your phone calls in real time.
Sure, the storage requirements would be crazy, but I don't see specific
details on what is to be stored, so it may well be implementable given
enough funding.
Permalink | Reply |

Gmane