Linux on the console

Microcai | 28 Nov 2010 11:57

VT console need rewrite

Hi, there

	I'm implementing the UNICODE font of the framebuffer console, (see
http://lkml.org/lkml/2010/11/26/50 in case you do not got my email). But
current vt code is too bugy, too many direct assumes about vt buffer,
This makes me so hard to hack.  There is TODO telling me to add UNICODE
support, but no room for such code, that's why my patch is so tricky.

	And the code itself, if you'll excuse me, it isn't as beautiful as rest
of the kernel.
	So, it really really need a clean rewrite.I'm ganna take is hard job.	
	And, please tell me if is worth to do so.

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

Theodore Tso | 28 Nov 2010 14:24

Re: VT console need rewrite

On Nov 28, 2010, at 5:57 AM, Microcai wrote:

> Hi, there
> 
> 	I'm implementing the UNICODE font of the framebuffer console, (see
> http://lkml.org/lkml/2010/11/26/50 in case you do not got my email). But
> current vt code is too bugy, too many direct assumes about vt buffer,
> This makes me so hard to hack.  There is TODO telling me to add UNICODE
> support, but no room for such code, that's why my patch is so tricky.
> 
> 	And the code itself, if you'll excuse me, it isn't as beautiful as rest
> of the kernel.
> 	So, it really really need a clean rewrite.I'm ganna take is hard job.	
> 	And, please tell me if is worth to do so.

Yes, the console is code is very old.   But please be aware that lots of code (both in the kernel and in
userspace) has dependencies upon how the code behaves.   So changing it in a way that does not break
backwards compatibility is hard.  i.e., it is hard to hack for a reason.

I would recommend an incremental rewrite (i.e., one patch at a time), as opposed to a rewrite from scratch.  
Because people will want to be assured that things haven't broken in a horrible way as a result of a complete rewrite...

-- Ted

headers

Microcai | 28 Nov 2010 14:42

Re: VT console need rewrite

在 2010-11-28日的 08:24 -0500，Theodore Tso写道：
> On Nov 28, 2010, at 5:57 AM, Microcai wrote:
> 
> > Hi, there
> > 
> > 	I'm implementing the UNICODE font of the framebuffer console, (see
> > http://lkml.org/lkml/2010/11/26/50 in case you do not got my email). But
> > current vt code is too bugy, too many direct assumes about vt buffer,
> > This makes me so hard to hack.  There is TODO telling me to add UNICODE
> > support, but no room for such code, that's why my patch is so tricky.
> > 
> > 	And the code itself, if you'll excuse me, it isn't as beautiful as rest
> > of the kernel.
> > 	So, it really really need a clean rewrite.I'm ganna take is hard job.	
> > 	And, please tell me if is worth to do so.
> 
> Yes, the console is code is very old.   But please be aware that lots of code (both in the kernel and in
userspace) has dependencies upon how the code behaves.   So changing it in a way that does not break
backwards compatibility is hard.  i.e., it is hard to hack for a reason.
> 
> I would recommend an incremental rewrite (i.e., one patch at a time), as opposed to a rewrite from scratch.  
Because people will want to be assured that things haven't broken in a horrible way as a result of a complete rewrite...
> 
> -- Ted
> 

Yeah, I'd also like to rewrite it incrementally. But... who will accept
that incrementally patch ? It just seems that incremental patch will be
horrible at the beginning...... It will be discard without a
reason .....

(Continue reading)

headers

jonsmirl@gmail.com | 28 Nov 2010 15:05

Re: VT console need rewrite

On Sun, Nov 28, 2010 at 8:42 AM, Microcai <microcai <at> fedoraproject.org> wrote:
> 在 2010-11-28日的 08:24 -0500，Theodore Tso写道：
>> On Nov 28, 2010, at 5:57 AM, Microcai wrote:
>>
>> > Hi, there
>> >
>> >     I'm implementing the UNICODE font of the framebuffer console, (see
>> > http://lkml.org/lkml/2010/11/26/50 in case you do not got my email). But
>> > current vt code is too bugy, too many direct assumes about vt buffer,
>> > This makes me so hard to hack.  There is TODO telling me to add UNICODE
>> > support, but no room for such code, that's why my patch is so tricky.
>> >
>> >     And the code itself, if you'll excuse me, it isn't as beautiful as rest
>> > of the kernel.
>> >     So, it really really need a clean rewrite.I'm ganna take is hard job.
>> >     And, please tell me if is worth to do so.
>>
>> Yes, the console is code is very old.   But please be aware that lots of code (both in the kernel and in
userspace) has dependencies upon how the code behaves.   So changing it in a way that does not break
backwards compatibility is hard.  i.e., it is hard to hack for a reason.
>>
>> I would recommend an incremental rewrite (i.e., one patch at a time), as opposed to a rewrite from
scratch.   Because people will want to be assured that things haven't broken in a horrible way as a result
of a complete rewrite...
>>
>> -- Ted
>>
>
> Yeah, I'd also like to rewrite it incrementally. But... who will accept
> that incrementally patch ? It just seems that incremental patch will be

(Continue reading)

headers

Microcai | 28 Nov 2010 15:29

Re: VT console need rewrite

在 2010-11-28日的 09:05 -0500，jonsmirl <at> gmail.com写道：
> On Sun, Nov 28, 2010 at 8:42 AM, Microcai <microcai <at> fedoraproject.org> wrote:
> > 在 2010-11-28日的 08:24 -0500，Theodore Tso写道：
> >> On Nov 28, 2010, at 5:57 AM, Microcai wrote:
> >>
> >> > Hi, there
> >> >
> >> >     I'm implementing the UNICODE font of the framebuffer console, (see
> >> > http://lkml.org/lkml/2010/11/26/50 in case you do not got my email). But
> >> > current vt code is too bugy, too many direct assumes about vt buffer,
> >> > This makes me so hard to hack.  There is TODO telling me to add UNICODE
> >> > support, but no room for such code, that's why my patch is so tricky.
> >> >
> >> >     And the code itself, if you'll excuse me, it isn't as beautiful as rest
> >> > of the kernel.
> >> >     So, it really really need a clean rewrite.I'm ganna take is hard job.
> >> >     And, please tell me if is worth to do so.
> >>
> >> Yes, the console is code is very old.   But please be aware that lots of code (both in the kernel and in
userspace) has dependencies upon how the code behaves.   So changing it in a way that does not break
backwards compatibility is hard.  i.e., it is hard to hack for a reason.
> >>
> >> I would recommend an incremental rewrite (i.e., one patch at a time), as opposed to a rewrite from
scratch.   Because people will want to be assured that things haven't broken in a horrible way as a result of a
complete rewrite...
> >>
> >> -- Ted
> >>
> >
> > Yeah, I'd also like to rewrite it incrementally. But... who will accept

(Continue reading)

headers

jonsmirl@gmail.com | 28 Nov 2010 15:49

Re: VT console need rewrite

On Sun, Nov 28, 2010 at 9:29 AM, Microcai <microcai <at> fedoraproject.org> wrote:
> 在 2010-11-28日的 09:05 -0500，jonsmirl <at> gmail.com写道：
>> On Sun, Nov 28, 2010 at 8:42 AM, Microcai <microcai <at> fedoraproject.org> wrote:
>> > 在 2010-11-28日的 08:24 -0500，Theodore Tso写道：
>> >> On Nov 28, 2010, at 5:57 AM, Microcai wrote:
>> >>
>> >> > Hi, there
>> >> >
>> >> >     I'm implementing the UNICODE font of the framebuffer console, (see
>> >> > http://lkml.org/lkml/2010/11/26/50 in case you do not got my email). But
>> >> > current vt code is too bugy, too many direct assumes about vt buffer,
>> >> > This makes me so hard to hack.  There is TODO telling me to add UNICODE
>> >> > support, but no room for such code, that's why my patch is so tricky.
>> >> >
>> >> >     And the code itself, if you'll excuse me, it isn't as beautiful as rest
>> >> > of the kernel.
>> >> >     So, it really really need a clean rewrite.I'm ganna take is hard job.
>> >> >     And, please tell me if is worth to do so.
>> >>
>> >> Yes, the console is code is very old.   But please be aware that lots of code (both in the kernel and in
userspace) has dependencies upon how the code behaves.   So changing it in a way that does not break
backwards compatibility is hard.  i.e., it is hard to hack for a reason.
>> >>
>> >> I would recommend an incremental rewrite (i.e., one patch at a time), as opposed to a rewrite from
scratch.   Because people will want to be assured that things haven't broken in a horrible way as a result
of a complete rewrite...
>> >>
>> >> -- Ted
>> >>
>> >

(Continue reading)

headers

Samuel Thibault | 28 Nov 2010 20:11

Re: VT console need rewrite

Microcai, le Sun 28 Nov 2010 22:29:05 +0800, a écrit :
> So, here is the design according to your description  .. 
> 
> JUST export /dev/fb and /dev/pts and /dev/console
> The kernel use /dev/console, but invisible. 
> The rest of the world uses /dev/pts
> 
> Make init using /dev/fb to display boot message, and agetty runs
> on /dev/pts

Errr, but that already exists, that's fbterm...

Samuel

headers

Microcai | 29 Nov 2010 02:10

Re: VT console need rewrite

在 2010-11-28日的 20:11 +0100，Samuel Thibault写道：
> Microcai, le Sun 28 Nov 2010 22:29:05 +0800, a écrit :
> > So, here is the design according to your description  .. 
> > 
> > JUST export /dev/fb and /dev/pts and /dev/console
> > The kernel use /dev/console, but invisible. 
> > The rest of the world uses /dev/pts
> > 
> > Make init using /dev/fb to display boot message, and agetty runs
> > on /dev/pts
> 
> Errr, but that already exists, that's fbterm...
> 
> Samuel

Every time someone try to implement something , some other guy will jump
out and say, that already exist, in user-space ..... blablablabla

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

Américo Wang | 29 Nov 2010 03:49

Re: VT console need rewrite

On Mon, Nov 29, 2010 at 09:10:06AM +0800, Microcai wrote:
>在 2010-11-28日的 20:11 +0100，Samuel Thibault写道：
>> Microcai, le Sun 28 Nov 2010 22:29:05 +0800, a écrit :
>> > So, here is the design according to your description  .. 
>> > 
>> > JUST export /dev/fb and /dev/pts and /dev/console
>> > The kernel use /dev/console, but invisible. 
>> > The rest of the world uses /dev/pts
>> > 
>> > Make init using /dev/fb to display boot message, and agetty runs
>> > on /dev/pts
>> 
>> Errr, but that already exists, that's fbterm...
>> 
>> Samuel
>
>Every time someone try to implement something , some other guy will jump
>out and say, that already exist, in user-space ..... blablablabla
>

Remember, remember, kernel provides a mechanism, not a policy.
(From this point, auto-groups *does* break it.)
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

Microcai | 29 Nov 2010 03:48

Re: VT console need rewrite

在 2010-11-29一的 10:49 +0800，Américo Wang写道：
> On Mon, Nov 29, 2010 at 09:10:06AM +0800, Microcai wrote:
> >在 2010-11-28日的 20:11 +0100，Samuel Thibault写道：
> >> Microcai, le Sun 28 Nov 2010 22:29:05 +0800, a écrit :
> >> > So, here is the design according to your description  .. 
> >> > 
> >> > JUST export /dev/fb and /dev/pts and /dev/console
> >> > The kernel use /dev/console, but invisible. 
> >> > The rest of the world uses /dev/pts
> >> > 
> >> > Make init using /dev/fb to display boot message, and agetty runs
> >> > on /dev/pts
> >> 
> >> Errr, but that already exists, that's fbterm...
> >> 
> >> Samuel
> >
> >Every time someone try to implement something , some other guy will jump
> >out and say, that already exist, in user-space ..... blablablabla
> >
> 
> Remember, remember, kernel provides a mechanism, not a policy.
> (From this point, auto-groups *does* break it.)

YES, but handle UNICODE font is a policy????????

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

(Continue reading)

headers

Samuel Thibault | 29 Nov 2010 09:16

Re: VT console need rewrite

Microcai, le Mon 29 Nov 2010 09:10:06 +0800, a écrit :
> 在 2010-11-28日的 20:11 +0100，Samuel Thibault写道：
> > Microcai, le Sun 28 Nov 2010 22:29:05 +0800, a écrit :
> > > So, here is the design according to your description  .. 
> > > 
> > > JUST export /dev/fb and /dev/pts and /dev/console
> > > The kernel use /dev/console, but invisible. 
> > > The rest of the world uses /dev/pts
> > > 
> > > Make init using /dev/fb to display boot message, and agetty runs
> > > on /dev/pts
> > 
> > Errr, but that already exists, that's fbterm...
> 
> Every time someone try to implement something , some other guy will jump
> out and say, that already exist, in user-space ..... blablablabla

But isn't that basically what you are describing?

(i.e. what's the use of using a pts and /dev/fb, if not for a userlevel
terminal to actually do the work?)

Samuel
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

microcai | 29 Nov 2010 09:20

Re: VT console need rewrite

> But isn't that basically what you are describing?
>
> (i.e. what's the use of using a pts and /dev/fb, if not for a userlevel
> terminal to actually do the work?)
>

ok, try run zhcon on pts.  can you do that ?
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

Samuel Thibault | 29 Nov 2010 09:27

Re: VT console need rewrite

microcai, le Mon 29 Nov 2010 16:20:11 +0800, a écrit :
> > But isn't that basically what you are describing?
> >
> > (i.e. what's the use of using a pts and /dev/fb, if not for a userlevel
> > terminal to actually do the work?)
> >
> 
> ok, try run zhcon on pts.  can you do that ?

zhcon _provides_ a pts, that's what I mean.

Samuel

headers

Microcai | 29 Nov 2010 09:53

Re: VT console need rewrite

在 2010-11-29一的 09:27 +0100，Samuel Thibault写道：
> microcai, le Mon 29 Nov 2010 16:20:11 +0800, a écrit :
> > > But isn't that basically what you are describing?
> > >
> > > (i.e. what's the use of using a pts and /dev/fb, if not for a userlevel
> > > terminal to actually do the work?)
> > >
> > 
> > ok, try run zhcon on pts.  can you do that ?
> 
> zhcon _provides_ a pts, that's what I mean.

So you agree zhcon can't run on pts.
Then there's dozens of zhcon.

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

Samuel Thibault | 29 Nov 2010 09:59

Re: VT console need rewrite

Microcai, le Mon 29 Nov 2010 16:53:25 +0800, a écrit :
> 在 2010-11-29一的 09:27 +0100，Samuel Thibault写道：
> > microcai, le Mon 29 Nov 2010 16:20:11 +0800, a écrit :
> > > > But isn't that basically what you are describing?
> > > >
> > > > (i.e. what's the use of using a pts and /dev/fb, if not for a userlevel
> > > > terminal to actually do the work?)
> > > >
> > > 
> > > ok, try run zhcon on pts.  can you do that ?
> > 
> > zhcon _provides_ a pts, that's what I mean.
> 
> So you agree zhcon can't run on pts.

Sure. It doesn't make sense. But what zhcon does should be done in
fbterm.

> Then there's dozens of zhcon.

What do you mean by this?

Samuel
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

microcai | 29 Nov 2010 09:20

Re: VT console need rewrite

> But isn't that basically what you are describing?
>
> (i.e. what's the use of using a pts and /dev/fb, if not for a userlevel
> terminal to actually do the work?)
>

ok, try run zhcon on pts.  can you do that ?

headers

Lisa Milne | 28 Nov 2010 17:06

Re: VT console need rewrite

On Sun, 28 Nov 2010 09:05:44 -0500
"jonsmirl <at> gmail.com" <jonsmirl <at> gmail.com> wrote:

> On Sun, Nov 28, 2010 at 8:42 AM, Microcai
> <microcai <at> fedoraproject.org> wrote:
> >
> > Yeah, I'd also like to rewrite it incrementally. But... who will
> > accept that incrementally patch ? It just seems that incremental
> > patch will be horrible at the beginning...... It will be discard
> > without a reason .....
> 
> You can use CONFIG_VT to remove the entire VT subsystem. It might be
> easier for you to write an alternative VT system that could be enabled
> with a different flag.
> 
> The VT system is very old code from the earliest days of Linux.
> Thousands of things depend on it both in the kernel and user space. It
> will be very hard to make significant changes to it that don't break
> lots of dependent code.
> 
> Another model to consider... Remove the VT subsystem. Replace it will
> a Unicode VT system built in user space. Using the existing kernel
> code, leave a single user console in the kernel that would only be
> used for system maintenance. Normal users would never see this console
> unless their system was really messed up.

Another possible model: split the current system in 2, so there's a
bytestream handler, and a vt-legacy module. Then use the interface
between bytestream/legacy as an interface for future vt-kernel and
vt-user modules.

(Continue reading)

headers

Microcai | 28 Nov 2010 17:20

Re: VT console need rewrite


> 
> Another possible model: split the current system in 2, so there's a
> bytestream handler, and a vt-legacy module. Then use the interface
> between bytestream/legacy as an interface for future vt-kernel and
> vt-user modules.

this may cause early printk stop working. 

> 
> This may make it possible to create an initial patch to do the split,
> then work on the new system independently of the current vt system.
> Hopefully reducing any problems with api/subsystem inconsistencies
> breaking existing code elsewhere, by giving it time to adapt.
> 
> This is guesswork on my part as I haven't actually looked at the code,
> so while it sounds good in theory, you'd have to check if it's actually
> doable.
> 

Sounds like a good idea. Who is in charge of VT system ? Seems no
one ....

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

Ted Ts'o | 28 Nov 2010 20:46

Re: VT console need rewrite

On Mon, Nov 29, 2010 at 12:20:07AM +0800, Microcai wrote:
> 
> > Another possible model: split the current system in 2, so there's a
> > bytestream handler, and a vt-legacy module. Then use the interface
> > between bytestream/legacy as an interface for future vt-kernel and
> > vt-user modules.
> 
> this may cause early printk stop working. 

Let's start by asking a much more fundamental question; what the heck
are your goals?

If the main goal of the console is emergency debugging when the system
is in a very bad state (i.e., trashed initrd, etc.) do we really even
need Unicode support?

How many people do regular login, development, reading e-mail, etc.,
on the console?  Very few!  If the answer is because you hate X, as
you've already pointed out, we already have fbterm.  Where is it
written that we need to have a full unicode-capable console system?
Why is this so important; especially if doing this is going to be very
difficult, and risks breaking lots of stuff if we try to mess with it?

	       	     	      	      - Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

(Continue reading)

headers

Microcai | 29 Nov 2010 02:14

Re: VT console need rewrite

在 2010-11-28日的 14:46 -0500，Ted Ts'o写道：
> On Mon, Nov 29, 2010 at 12:20:07AM +0800, Microcai wrote:
> > 
> > > Another possible model: split the current system in 2, so there's a
> > > bytestream handler, and a vt-legacy module. Then use the interface
> > > between bytestream/legacy as an interface for future vt-kernel and
> > > vt-user modules.
> > 
> > this may cause early printk stop working. 
> 
> Let's start by asking a much more fundamental question; what the heck
> are your goals?
> 
> If the main goal of the console is emergency debugging when the system
> is in a very bad state (i.e., trashed initrd, etc.) do we really even
> need Unicode support?
> 
> How many people do regular login, development, reading e-mail, etc.,
> on the console?  Very few!  If the answer is because you hate X, as
> you've already pointed out, we already have fbterm.  Where is it
> written that we need to have a full unicode-capable console system?
> Why is this so important; especially if doing this is going to be very
> difficult, and risks breaking lots of stuff if we try to mess with it?
> 
> 	       	     	      	      - Ted

Hey, my old patch already did it , and do not break any old stuff.
Question is , the VT code *is really very old*. 
Just want to simplify the code, remove old stuff, make it future
compatible. Forward compatibility is more important than backward one

(Continue reading)

headers

Samuel Thibault | 29 Nov 2010 09:58

Re: VT console need rewrite

Microcai, le Mon 29 Nov 2010 09:14:04 +0800, a écrit :
> Hey, my old patch already did it , and do not break any old stuff.

To your knowledge.  IIRC while reading your patch I found a couple of
things that would most probably break some corner cases, which could
go unnoticed, that's why I proposed to first extend the internal vc
content matrix into unicode values, so as to clearly have to upgrade all
drivers.

Samuel
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

Microcai | 29 Nov 2010 10:32

Re: VT console need rewrite

在 2010-11-29一的 09:58 +0100，Samuel Thibault写道：
> Microcai, le Mon 29 Nov 2010 09:14:04 +0800, a écrit :
> > Hey, my old patch already did it , and do not break any old stuff.
> 
> To your knowledge.  IIRC while reading your patch I found a couple of
> things that would most probably break some corner cases, which could
> go unnoticed, that's why I proposed to first extend the internal vc
> content matrix into unicode values, so as to clearly have to upgrade all
> drivers.
> 
> Samuel

I doesn't matter *how to do* . I have many different kinds of patchs
that did it in different ways.

What realy matters is that, do we agree that add unicode font support is
acceptable ?

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

Samuel Thibault | 29 Nov 2010 10:58

Re: VT console need rewrite

Microcai, le Mon 29 Nov 2010 17:32:37 +0800, a écrit :
> 在 2010-11-29一的 09:58 +0100，Samuel Thibault写道：
> > Microcai, le Mon 29 Nov 2010 09:14:04 +0800, a écrit :
> > > Hey, my old patch already did it , and do not break any old stuff.
> > 
> > To your knowledge.  IIRC while reading your patch I found a couple of
> > things that would most probably break some corner cases, which could
> > go unnoticed, that's why I proposed to first extend the internal vc
> > content matrix into unicode values, so as to clearly have to upgrade all
> > drivers.
> 
> I doesn't matter *how to do* .

Concerning breaking existing stuff, yes, it does.

> I have many different kinds of patchs that did it in different ways.
> 
> What realy matters is that, do we agree that add unicode font support is
> acceptable ?

Simple unicode font (i.e. just go beyond the arbitrary historical-vga
512 glyphs limit for fbcon) might go in.  Double-width is questionable,
but it can probably simple enough to be able to get in (there is already
some code for it anyway).

But complete unicode support (with arabic ligatures, tibetan
combinations, etc.) won't ever be accepted in the kernel as that's far
too involved in terms of font rendering.  So the userland way needs to
be fixed for these anyway, and then things like chinese will go along...

(Continue reading)

headers

Microcai | 29 Nov 2010 11:11

Re: VT console need rewrite

在 2010-11-29一的 10:58 +0100，Samuel Thibault写道：
> 
> Simple unicode font (i.e. just go beyond the arbitrary historical-vga
> 512 glyphs limit for fbcon) might go in.  Double-width is
> questionable,
> but it can probably simple enough to be able to get in (there is
> already
> some code for it anyway).
> 
> But complete unicode support (with arabic ligatures, tibetan
> combinations, etc.) won't ever be accepted in the kernel as that's far
> too involved in terms of font rendering.  So the userland way needs to
> be fixed for these anyway, and then things like chinese will go
> along...
> 
> Samuel 

YES, my patch just include BMP(mostly CJK), not full UNICODE. IF it can
be included, then I can do more work to make sure it doesn't break old
stuff. And it's configurable. users don't have to be forced to use.

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

Samuel Thibault | 29 Nov 2010 11:17

Re: VT console need rewrite

Microcai, le Mon 29 Nov 2010 18:11:16 +0800, a écrit :
> 在 2010-11-29一的 10:58 +0100，Samuel Thibault写道：
> > Simple unicode font (i.e. just go beyond the arbitrary historical-vga
> > 512 glyphs limit for fbcon) might go in.  Double-width is
> > questionable,
> > but it can probably simple enough to be able to get in (there is
> > already
> > some code for it anyway).
> > 
> > But complete unicode support (with arabic ligatures, tibetan
> > combinations, etc.) won't ever be accepted in the kernel as that's far
> > too involved in terms of font rendering.  So the userland way needs to
> > be fixed for these anyway, and then things like chinese will go
> > along...
> > 
> > Samuel 
> 
> YES, my patch just include BMP(mostly CJK), not full UNICODE. IF it can
> be included, then I can do more work to make sure it doesn't break old
> stuff. And it's configurable. users don't have to be forced to use.

It seems you didn't get my point: what is the use of including CJK into
kernel when that won't solve the issue for e.g. arabic, while fixing
fbterm will solve the issue for both arabic and CJK?

Samuel

headers

Microcai | 29 Nov 2010 11:40

Re: VT console need rewrite

在 2010-11-29一的 11:17 +0100，Samuel Thibault写道：
> It seems you didn't get my point: what is the use of including CJK
> into
> kernel when that won't solve the issue for e.g. arabic, while fixing
> fbterm will solve the issue for both arabic and CJK?
> 

You don't get my point.  fbterm of zhcon won't fit every thing.

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

Samuel Thibault | 29 Nov 2010 11:39

Re: VT console need rewrite

Microcai, le Mon 29 Nov 2010 18:40:00 +0800, a écrit :
> 在 2010-11-29一的 11:17 +0100，Samuel Thibault写道：
> > It seems you didn't get my point: what is the use of including CJK
> > into
> > kernel when that won't solve the issue for e.g. arabic, while fixing
> > fbterm will solve the issue for both arabic and CJK?
> 
> You don't get my point.  fbterm of zhcon won't fit every thing.

Why?  Can't what zhcon does be integrated into fbterm?

Samuel
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

Pavel Machek | 15 Dec 2010 15:45

Re: VT console need rewrite

Hi!

> > I have many different kinds of patchs that did it in different ways.
> > 
> > What realy matters is that, do we agree that add unicode font support is
> > acceptable ?
> 
> Simple unicode font (i.e. just go beyond the arbitrary historical-vga
> 512 glyphs limit for fbcon) might go in.  Double-width is questionable,

That would be very nice, yes.

> But complete unicode support (with arabic ligatures, tibetan
> combinations, etc.) won't ever be accepted in the kernel as that's far
> too involved in terms of font rendering.  So the userland way needs to
> be fixed for these anyway, and then things like chinese will go along...

Well... english has ligatures, too (fi); console does not support them
nicely and we still use it. Support for eastern european characters by
default would be very very nice... and yes, that's just breaking the
256 char limit, nothing more complex.
									Pavel
--

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

(Continue reading)

headers

Samuel Thibault | 15 Dec 2010 16:04

Re: VT console need rewrite

Pavel Machek, le Wed 15 Dec 2010 15:45:02 +0100, a écrit :
> > But complete unicode support (with arabic ligatures, tibetan
> > combinations, etc.) won't ever be accepted in the kernel as that's far
> > too involved in terms of font rendering.  So the userland way needs to
> > be fixed for these anyway, and then things like chinese will go along...
> 
> Well... english has ligatures, too (fi);

English ligatures are extremely far from what arabic needs. People
don't even actually know that there are ligatures in English, while for
arabic, people would even have a hard time reading something that isn't
using ligatures.

> console does not support them nicely and we still use it.

Err, I've very rarely seen the fi/ff ligatures on the console, it was
mostly from copy/pastes from pdf documents.

> Support for eastern european characters by default would be very very
> nice... and yes, that's just breaking the 256 char limit, nothing more
> complex.

The console can do 512 glyphs, not only 256, so eastern europe support
is a matter of combining a latin1 font with parts of the other latin
fonts. That already covers a lot of european languages.

Samuel
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org

(Continue reading)

headers

Pavel Machek | 15 Dec 2010 15:45

Re: VT console need rewrite

Hi!

> > I have many different kinds of patchs that did it in different ways.
> > 
> > What realy matters is that, do we agree that add unicode font support is
> > acceptable ?
> 
> Simple unicode font (i.e. just go beyond the arbitrary historical-vga
> 512 glyphs limit for fbcon) might go in.  Double-width is questionable,

That would be very nice, yes.

> But complete unicode support (with arabic ligatures, tibetan
> combinations, etc.) won't ever be accepted in the kernel as that's far
> too involved in terms of font rendering.  So the userland way needs to
> be fixed for these anyway, and then things like chinese will go along...

Well... english has ligatures, too (fi); console does not support them
nicely and we still use it. Support for eastern european characters by
default would be very very nice... and yes, that's just breaking the
256 char limit, nothing more complex.
									Pavel
--

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

headers

Alexey Gladkov | 29 Nov 2010 02:55

Re: VT console need rewrite

28.11.2010 22:46, Ted Ts'o wrote:
> How many people do regular login, development, reading e-mail, etc.,
> on the console?  Very few!

Once I broke the euro symbol in the kbd keymaps and got a kick in the
ass very quickly :). It means that users continue to use the console
quite active.

Another example: I keep getting a new keymaps and console fonts for
each new kbd release.

Maybe not a lot of console users, but more of them than you think.

> If the answer is because you hate X, as
> you've already pointed out, we already have fbterm.  Where is it
> written that we need to have a full unicode-capable console system?

fbterm ‎can't help in all cases. For example: we have a limitation
NR_KEYS, which makes it impossible to use keycode > 256 in keymaps.
I've already received several complaints about it.

--

-- 
Rgrds, legion

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

(Continue reading)

headers

Microcai | 29 Nov 2010 03:14

Re: VT console need rewrite

在 2010-11-29一的 04:55 +0300，Alexey Gladkov写道：
> 28.11.2010 22:46, Ted Ts'o wrote:
> > How many people do regular login, development, reading e-mail, etc.,
> > on the console?  Very few!
> 
> Once I broke the euro symbol in the kbd keymaps and got a kick in the
> ass very quickly :). It means that users continue to use the console
> quite active.
> 
> Another example: I keep getting a new keymaps and console fonts for
> each new kbd release.
> 
> Maybe not a lot of console users, but more of them than you think.
> 
> > If the answer is because you hate X, as
> > you've already pointed out, we already have fbterm.  Where is it
> > written that we need to have a full unicode-capable console system?
> 
> fbterm ‎can't help in all cases. For example: we have a limitation
> NR_KEYS, which makes it impossible to use keycode > 256 in keymaps.
> I've already received several complaints about it.
> 

YES, user-space can't always fit! Some function need to be implemented
in kernel. And that's why we have KMS, we have auto cgroup ....

Don't say, hey , hard to implement, all you have to to is apply the
path .. No need to do for you.

If the patch isn't as good as you expected, reject, I'll try better one.

(Continue reading)

headers

Alan Cox | 29 Nov 2010 15:02

Re: VT console need rewrite

> If the patch isn't as good as you expected, reject, I'll try better one.
> The thing is, you really should agree that, implementing UNICODE font in
> kernel isn't a bad idea.

Unicode is a good deal more than a font.

Alan
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

Valdis.Kletnieks | 29 Nov 2010 18:50

Re: VT console need rewrite

On Mon, 29 Nov 2010 10:14:06 +0800, Microcai said:

> The thing is, you really should agree that, implementing UNICODE font in
> kernel isn't a bad idea.

Let us know when you have implemented support for bidirectional
scripts that require compositing or half-spacing characters.  Without
blowing out the 8K kernel stack.

Doing it *right* is important:

http://gizmodo.com/382026/a-cellphones-missing-dot-kills-two-people-puts-three-more-in-jail

Two dead, three in jail.  All because of broken font support.  That's a good
reason to not even try unless you can do it *right*.

headers

Jerome Glisse | 29 Nov 2010 22:23

Re: VT console need rewrite

On Mon, Nov 29, 2010 at 12:50 PM,  <Valdis.Kletnieks <at> vt.edu> wrote:
> On Mon, 29 Nov 2010 10:14:06 +0800, Microcai said:
>
>> The thing is, you really should agree that, implementing UNICODE font in
>> kernel isn't a bad idea.
>
> Let us know when you have implemented support for bidirectional
> scripts that require compositing or half-spacing characters.  Without
> blowing out the 8K kernel stack.
>
> Doing it *right* is important:
>
> http://gizmodo.com/382026/a-cellphones-missing-dot-kills-two-people-puts-three-more-in-jail
>
> Two dead, three in jail.  All because of broken font support.  That's a good
> reason to not even try unless you can do it *right*.
>
>

Side point, KMS people would like to see the direct memory access to
fb buffer vanish, right now we are wasting vram to keep a front buffer
around for fb. So somethings like always using some callback to access
(write/read) to fb buffer.

Cheers,
Jerome Glisse
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

(Continue reading)

headers

Alan Cox | 29 Nov 2010 00:46

Re: VT console need rewrite

> Sounds like a good idea. Who is in charge of VT system ? Seems no
> one ....

Which bit - lots of people own different parts of it, because there is a
lot of it, and people own different sections of currently ongoing
projects. Plus the X DRM folks own the KMS framebuffers, various authors
own their own low level drivers for each card and so on.

I kind of own the drivers/tty/vt* bits.

I simply don't see how you are going to address the majority of the
worlds languages in kernel space, all the major ones are too complex and
pango is too big and complex to put in kernel as are the needed font
renderers. Most of the needed compositing accelerations are also not
directly exposed in KMS to the console driver as would be needed

Your other problem is mode setting, VESA is on its way out, a lot of
devices today, especially non X86 ones pretty much need X to do any kind
of video management beyond "here is how the firmware console was
configured", and indeed many embedded devices don't have an VT code
enabled at all, just a printk console and possibly ttyprintk, both of
which may well only be talking to the debug ports

Alan
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

(Continue reading)

headers

Microcai | 29 Nov 2010 02:28

Re: VT console need rewrite

在 2010-11-28日的 23:46 +0000，Alan Cox写道：
> Which bit - lots of people own different parts of it, because there is a
> lot of it, and people own different sections of currently ongoing
> projects. Plus the X DRM folks own the KMS framebuffers, various authors
> own their own low level drivers for each card and so on.
> 
> I kind of own the drivers/tty/vt* bits.

Don't you see the cosole code ? There is TODO that say , "hey , I need
UNICODE support..."

I'm not breaking old one to introduce new feature, but fix a little BUG
in the TODO list. 

> I simply don't see how you are going to address the majority of the
> worlds languages in kernel space, all the major ones are too complex and
> pango is too big and complex to put in kernel as are the needed font
> renderers. Most of the needed compositing accelerations are also not
> directly exposed in KMS to the console driver as would be needed

Simply drawing characters won't need any hw acceleration. It doesn't
matter if it is slow. KMS and Wayland will make more people work under
console, trust me.

> 
> Your other problem is mode setting, VESA is on its way out, a lot of
No, VESA will still be here for another decades. Some fb driver conflict
with the X one, even there is a fb driver for you card, you simply won't
use it unless it does not conflict with X one. My PC , for example,
nvidiafb conflict with X one, so I have to use VESA.

(Continue reading)

headers

Alan Cox | 29 Nov 2010 15:07

Re: VT console need rewrite

> Don't you see the cosole code ? There is TODO that say , "hey , I need
> UNICODE support..."

And what has happened over time is that we've learned

- that real unicode is hard
- that some languages need very complex character handling, and this
  happens to be most of the major non western european languages in the
  world today

and someone put all the needed logic into pango, and people wrote fbterm
and friends. The world has moved on

> Simply drawing characters won't need any hw acceleration. It doesn't
> matter if it is slow. KMS and Wayland will make more people work under
> console, trust me.

No I don't trust you, sorry. You need compositing acceleration to do some
of the character composing efficiently and to do anti-aliasing which for
small symbols in many non western fonts is rather essential.

> > Your other problem is mode setting, VESA is on its way out, a lot of
> No, VESA will still be here for another decades. Some fb driver conflict

VESA is X86 specific, PC class hardware specific and subject to various
assumptions, when it works.

No VESA on Intel MID platforms, no VESA on PPC, no VESA on Android phones

> printk console , if redirect to a ttyS? , then the outer Linux world

(Continue reading)

headers

microcai | 30 Nov 2010 06:05

Re: VT console need rewrite

OK, let me vanish in a few months.

hopefully I'll be back with a patch fix that stupid 512 glyph limitation.
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

Pavel Machek | 20 Dec 2010 07:35

Re: VT console need rewrite

On Tue 2010-11-30 13:05:51, microcai wrote:
> OK, let me vanish in a few months.
> 
> hopefully I'll be back with a patch fix that stupid 512 glyph limitation.

Looking forward.
								Pavel
(Who'd rather like to see reasonable support for eastern *and* western
europe.)

--

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

Andi Kleen | 29 Nov 2010 11:01

Re: VT console need rewrite

Microcai <microcai <at> fedoraproject.org> writes:

> 	I'm implementing the UNICODE font of the framebuffer console, (see
> http://lkml.org/lkml/2010/11/26/50 in case you do not got my email). But
> current vt code is too bugy, too many direct assumes about vt buffer,
> This makes me so hard to hack.  There is TODO telling me to add UNICODE
> support, but no room for such code, that's why my patch is so tricky.
>
> 	And the code itself, if you'll excuse me, it isn't as beautiful as rest
> of the kernel.
> 	So, it really really need a clean rewrite.I'm ganna take is hard job.	
> 	And, please tell me if is worth to do so.

To be honest I have doubts its worth it. This was discussed years and
years ago and the conclusion back then that people who need unicode
fonts should use them in X or another graphical console environment
(there are plenty to chose from these days). Has this changed?

-Andi

--

-- 
ak <at> linux.intel.com -- Speaking for myself only.
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

Microcai | 29 Nov 2010 11:12

Re: VT console need rewrite

在 2010-11-29一的 11:01 +0100，Andi Kleen写道：

> To be honest I have doubts its worth it. This was discussed years and
> years ago and the conclusion back then that people who need unicode
> fonts should use them in X or another graphical console environment
> (there are plenty to chose from these days). Has this changed?
> 
> -Andi
> 

Yes, it change. Linux graphical stuck have changed a lot. With KMS,
console got a native resolution, and looks very nice.

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

headers

Jesse Barnes | 7 Dec 2010 21:37

Re: VT console need rewrite

On Mon, 29 Nov 2010 18:12:57 +0800
Microcai <microcai <at> fedoraproject.org> wrote:

> 在 2010-11-29一的 11:01 +0100，Andi Kleen写道：
> 
> > To be honest I have doubts its worth it. This was discussed years and
> > years ago and the conclusion back then that people who need unicode
> > fonts should use them in X or another graphical console environment
> > (there are plenty to chose from these days). Has this changed?
> > 
> > -Andi
> > 
> 
> Yes, it change. Linux graphical stuck have changed a lot. With KMS,
> console got a native resolution, and looks very nice.

But really, keep this stuff out of the kernel.  Sure, the kernel can
provide memory and a framebuffer to draw in (subject to the constraints
Jerome already mentioned), but why try to deal with fancy input or font
handling there?  IMO the kernel console should be limited to printing
debug and error information; a separate userspace daemon should handle
input and console support.  It would be great to avoid the need for
CONFIG_VT entirely for the typical Linux system.

--

-- 
Jesse Barnes, Intel Open Source Technology Center

headers

Pavel Machek | 13 Dec 2010 08:13

Re: VT console need rewrite

Hi!

> > 	And the code itself, if you'll excuse me, it isn't as beautiful as rest
> > of the kernel.
> > 	So, it really really need a clean rewrite.I'm ganna take is hard job.	
> > 	And, please tell me if is worth to do so.
> 
> To be honest I have doubts its worth it. This was discussed years and
> years ago and the conclusion back then that people who need unicode
> fonts should use them in X or another graphical console environment
> (there are plenty to chose from these days). Has this changed?

Yes. These days kernel sources and kernel messages both contain
non-ASCII characters... as well as many filesystems.
								Pavel

--

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

headers

Alan Cox | 13 Dec 2010 11:18

Re: VT console need rewrite

> Yes. These days kernel sources and kernel messages both contain
> non-ASCII characters... as well as many filesystems.

But not in kernel printed messages.

Alan

headers

Pavel Machek | 15 Dec 2010 15:40

Re: VT console need rewrite

Hi!

> > Yes. These days kernel sources and kernel messages both contain
> > non-ASCII characters... as well as many filesystems.
> 
> But not in kernel printed messages.

Yes, even in those (but something eats my mail).

CAFE driver has non-ascii characters in pci device name, and that
leaks to dmesg.
									Pavel
--

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html