WebKit open source development ()

headers Alpha Lam | 13 Aug 2012 20:08

_______________________________________________
webkit-dev mailing list
webkit-dev <at> lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev

_______________________________________________
webkit-dev mailing list
webkit-dev <at> lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev

_______________________________________________
webkit-dev mailing list
webkit-dev <at> lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev

_______________________________________________
webkit-dev mailing list
webkit-dev <at> lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev

_______________________________________________
webkit-dev mailing list
webkit-dev <at> lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev

2012/8/14 Alpha Lam <hclam <at> chromium.org>:
>
>
> 2012/8/13 KwangYul Seo <skyul <at> company100.net>
>>>
>>>
>>> This approach is probably safe (as far as I know) but would have the
>>> downside of an extra pass over the whole render tree, or else overhead of
>>> maintaining an up-to-date list of rendered images; and it would happen very
>>> close to painting.
>>>
>
> I agree on getting actual numbers. I suspect there will be some gain though
> the limiting factor is the slowest-to-decode image in the viewport.
>
>>
>> Unfortunately, yes. Because of the lazy nature of image decoding, we don't
>> have much time for decoding before images need to be painted. So we skip
>> image decoding (and just trigger decoding in the background) in the first
>> paint pass and update the decoded images later.
>>
>>>
>>>
>
> I looked at the demo video and honestly I don't find experience particularly
> bad with your approach. The argument here is that when a page is fully
> cached and images will be painted with an instantaneous white region, I
> think though caching is an implementation detail of both WebCore and the
> browser so user doesn't really know if a page is 100% cached. Think about an
> evicted image would have a similar effect.

I agree with you.

>
> I think the expectation is different for web (e.g. http, https) and always
> available content (e.g. file, data uri) from a developer / user's
> perspective.
>

I also wondered whether flashing of file or data uri contents is
acceptable. In my opinion, it might be acceptable, because it is hard
to recognize flashing in these cases too. If flashing is really
unacceptable, we can turn off parallel image decoder in these case.

DS Huang

> Alpha
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev <at> lists.webkit.org
> http://lists.webkit.org/mailman/listinfo/webkit-dev
>

2012/8/14 KwangYul Seo <skyul <at> company100.net>:
>
>
> On Tue, Aug 14, 2012 at 4:03 AM, Maciej Stachowiak <mjs <at> apple.com> wrote:
>>
>>
>> On Aug 13, 2012, at 11:08 AM, Alpha Lam <hclam <at> chromium.org> wrote:
>>
>> That's a good point. I'm not sure but a safe bet would be after RenderView
>> is layout'ed then iterate through images to start decoding.
>>
>> The same thing would be needed for scrolling too, as page scrolls need to
>> iterate images in the viewport.
>>
>>
>> This approach is probably safe (as far as I know) but would have the
>> downside of an extra pass over the whole render tree, or else overhead of
>> maintaining an up-to-date list of rendered images; and it would happen very
>> close to painting.
>>
>
> Yes, the problem is that we don't have much time between layout and painting
> because painting is usually  immediately followed by layout. I'm skeptical
> about the gain we could get by triggering image decoding in the layout
> phase, but I'd like to defer the conclusion until we have real numbers.

Trigger image decoding early in layout and scroll will certainly
relieve flashing though it can’t completely remove the problem. We
will apply this optimization to parallel image decoder.

Please note that this optimization is applied only to parallel image
decoder, not Google’s deferred image decoder.

Thank all for good information.

Regards,
DS Huang

>
>> I think to some extent, benefit from parallelism is in direct tension with
>> benefit from lazy on-demand decoding.
>>
>
> Unfortunately, yes. Because of the lazy nature of image decoding, we don't
> have much time for decoding before images need to be painted. So we skip
> image decoding (and just trigger decoding in the background) in the first
> paint pass and update the decoded images later.
>
>>
>> Note by the way that in general many kinds of render objects can result in
>> rendering images, not just RenderImage. Consider the various forms of CSS
>> images (CSS backgrounds, CSS content property pointing to an image, etc).
>>
>
> We made CSS background, border and mask images be decoded by parallel image
> decoders.
>
> https://bugs.webkit.org/show_bug.cgi?id=91203
>
>>
>> Regards,
>> Maciej
>>
>>
>> 2012/8/13 Alpha (Hin-Chung) Lam <hclam <at> google.com>
>>>
>>> That's a good point. I'm not sure but a safe bet would be after
>>> RenderView is layout'ed then iterate through images to start decoding.
>>>
>>> The same thing would be needed for scrolling too, as page scrolls need to
>>> iterate images in the viewport.
>>>
>>> Alpha
>>>
>>> 2012/8/13 Maciej Stachowiak <mjs <at> apple.com>
>>>>
>>>>
>>>> The thing I'm not confident of is whether an image's position in
>>>> absolute coordinates can be changed by an ancestor after RenderImage::layout
>>>> completes. It would be helpful if a layout expert would weigh in.
>>>>
>>>>  - Maciej
>>>>
>>>> On Aug 13, 2012, at 3:20 AM, Dong Seong Hwang <luxtella <at> company100.net>
>>>> wrote:
>>>>
>>>> > https://bugs.webkit.org/show_bug.cgi?id=90375#c80
>>>> >
>>>> > In the above link, Hin-Chung shows how to determine whether an image
>>>> > is actually painted.
>>>> >
>>>> > 2012/8/13 Maciej Stachowiak <mjs <at> apple.com>:
>>>> >> I that case, starting async decoding at layout time makes sense if
>>>> >> and only
>>>> >> if at layout to e you can predict what you will paint. I don't know
>>>> >> enough
>>>> >> about our rendering to know of that is the case.
>>>> >>
>>>> >> - Maciej
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Aug 12, 2012, at 5:34 PM, Peter Kasting <pkasting <at> chromium.org>
>>>> >> wrote:
>>>> >>
>>>> >> On Sun, Aug 12, 2012 at 1:24 PM, Maciej Stachowiak <mjs <at> apple.com>
>>>> >> wrote:
>>>> >>>
>>>> >>> Why not start asynchronous decoding immediately as the image is
>>>> >>> loading,
>>>> >>> and synchronize at paint time? What is the benefit of waiting until
>>>> >>> layout
>>>> >>> time to start decoding the image data?
>>>> >>
>>>> >>
>>>> >> Uninformed guess (since I haven't touched the decoders in a while),
>>>> >> but
>>>> >> currently we don't decode unless the image is actually painted, which
>>>> >> helps
>>>> >> a ton on pages that are an enormous long string of images (common
>>>> >> cases:
>>>> >> Boston Big Picture blog, various porn sites), since most of the
>>>> >> images can
>>>> >> be decoded after initial layout, or not at all (if the user never
>>>> >> scrolls
>>>> >> down enough).  If we started decoding as images loaded I'd imagine
>>>> >> we'd do
>>>> >> (possibly a lot of) extra work compared to today.
>>>> >>
>>>> >> PK
>>>> >>
>>>> >>
>>>> >> _______________________________________________
>>>> >> webkit-dev mailing list
>>>> >> webkit-dev <at> lists.webkit.org
>>>> >> http://lists.webkit.org/mailman/listinfo/webkit-dev
>>>> >>
>>>>
>>>> _______________________________________________
>>>> webkit-dev mailing list
>>>> webkit-dev <at> lists.webkit.org
>>>> http://lists.webkit.org/mailman/listinfo/webkit-dev
>>>
>>>
>>
>>
>>
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev <at> lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo/webkit-dev
>>
>
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev <at> lists.webkit.org
> http://lists.webkit.org/mailman/listinfo/webkit-dev
>

2012/8/14 Dong Seong Hwang <luxtella <at> company100.net>
> Trigger image decoding early in layout and scroll will certainly
> relieve flashing though it can’t completely remove the problem. We
> will apply this optimization to parallel image decoder.
>

I changed parallel image decoder to trigger decoding on layout as
Alpha suggested. Unfortunately, it does not seem to relieve flashing.
I tested the change on Tumblr site, but the difference was visually
indistinguishable.

However, theoretically there must be some improvements. I want to
discuss more about it while parallel image decoder is reviewed.

_______________________________________________
webkit-dev mailing list
webkit-dev <at> lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev

2012/8/21 James Robinson <jamesr <at> google.com>:
>
> - James
>

After reading your writing, I could understand why Chromium team is
implementing deferred image decoder. I definitely agree on your
opinion that we cannot accept the implementation which worsens user
experience.

As mentioned above, we have two approaches to buy more time to decode
an image off the main thread.

We prototyped the approach which moves the start line forward. We had
to change CachedImage, RenderImage, RenderObject and so on, so that
parallel image decoder has the information to decide which image
should be decoded prior to other images. However, we finally gave up
this approach because it caused tight coupling among many modules and
could not ensure enough time to decode images off the main thread.

Consequently, deferred image decoder seems to be the best approach to
decode an image off the main thread. However, Google's deferred image
decoder seems to depend on SkPicture, so other ports can not reuse it.

My company, Company100 has focused on WebKit parallelization to
improve performance. We have a plan to implement recording and
replaying of GraphicsContext, which is similar to SkPicture of Skia or
recording surface of Cairo. Buffering GraphicsContext commands could
allow us a number of serial optimizations such as collapsing draws,
dropping fully obscured draw calls as you mentioned as well as
parallel optimizations such as parallel painting of each layer of
GraphicsLayer tree and off the main thread canvas rendering.

For now, we suggest that each platform decides whether to enable
parallel image decoder or not at the compile time using
ENABLE(PARALLEL_IMAGE_DECODER) guard.

Parallel image decoder can be removed from WebKit once we have a
platform independent deferred image decoder.

Regards,
DS Huang

Re: Status of multithreaded image decoding

Re: Status of multithreaded image decoding

Re: Status of multithreaded image decoding

Re: Status of multithreaded image decoding

Re: Status of multithreaded image decoding

Re: Status of multithreaded image decoding

Re: Status of multithreaded image decoding

Re: Status of multithreaded image decoding

Re: Status of multithreaded image decoding

Re: Status of multithreaded image decoding