1. Attendees
  2. Apologies
  3. Topics
  4. Frontends
  5. Forms and gadgets
  6. History navigation scroll
  7. Bug fixes
  8. Next time

Attendees

  • Michael Drake
  • Vincent Sanders
  • Daniel Silverstone

Apologies

  • John-Mark Bell
  • Chris Young

Topics

  • Logging merge for main NetSurf
    • Daniel added verbose and normal logging options
    • Vince added documentation for nslog in netsurf
    • Vince fixed a whole pile of horrible compile problems with nslog on atari etc.
    • Daniel fixed the test suite to work with the new logging options

  • Triage of bug tracker to generate further topics for Saturday

    • Michael found a couple of 10+ year old feature requests assigned to him, and decided it was time to implement them, so he got started on:
    • Daniel fixed #2549
    • Daniel reviewed #2537 and decided that it wasn't worth trying to fix now. It's reported against 3.6, the reported issue is fixed, but the underlying inability to interact with buttons is not.
    • Daniel and vince reviewed #2555 and decided François had already fixed it.
    • Daniel reviewed #2544 and decided it had always worked, closed it can't reproduce.
    • Daniel tried to reproduce #2532 but failed, so asked the submitter to try again.
    • Daniel looked at:
    • Daniel reviewed #526 and assigned it to Michael
    • Vince reviewed #2550 and closed it unable-to-reproduce.
    • Daniel and Vince worked through the following, closing most of them...

  • What frontends do we demote?

    • See below

  • Administrivia - Phoenix's disks are tired (even SMART said so)

    • We'd like to know where the society bank account stands, for general resources.
    • We'd like to buy a pair of WD Gold 1TB drives (rough cost 140 UKP total) so that Phoenix isn't running on pre-fail age drives.
    • We investigated cloud/hosted servers, sadly it's super-expensive compared to some new disks for the free hosting we have.
    • Decision was: Michael, Vince, and Daniel will share the cost if the society lacks the funds. Vince will buy drives and begin the replacement process. Ideally ASAP.

  • Discussion around forms, removing gadgets, etc.

    • See below

  • Auto-scroll to last position during history navigation

    • See below

  • Breaking changes and 4.x

    • We have defined 'major' changes as anything large (e.g. new layout would be major)
    • Also 'major' changes include on-disk format changes. This means that we'll forward-migrate where possible, but make no guarantees about going backwards. Migration will be explicit and cancellable on startup.
    • Content migrations must be such that if you run an old version, it won't pick up old content by default.

  • What's left to go in to 3.7

    • Michael's searching for hotlist (treeview search)
    • Vince's fixes for bitmaps in local history
    • History navigation keeping scroll offsets
    • Logging rework fallout
    • General memory leak fixes where possible
    • Aiming for 3.7 release handle crankage around Saturday September 23rd.

  • URLdb/Cookies

    • URLdb is a mess. We knew that, there's nothing new there.
    • We'd like to replace it, in its entirety, including on-disk formats.
    • It should be split into a proper cookie jar, and a metadata store keyed by URL.
    • For the cookie jar, there is an RFC for cookie handling, lots of useful information and links on https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies
    • For the "URL database" we need to store arbitrary and changeable sets of key-to-value mappings, some of which need to persist, some of which are to be RAM-only.
    • If we add scope to the 'key' such that we can have localstorage as a scope then we can put local storage https://www.w3.org/TR/webstorage/ into the same DB.
    • Potential implementation strategy:
      • Use a journalling approach (where the entries are create/change and delete commands) storing the journal as the URLdb replacement.
      • The primary key into the DB is based on nsurl.
      • We will reuse the resource: scheme for "internal" storage. (e.g. session state, disk cache indices etc)
      • The "value set" is a bag of triples. (scope, key, value) where the scope is one of an enumeration (e.g. localStorage or urldb) the key is a simple utf-8 string, and the value is an arbitrary set of bytes (up to 4GB). (thus is stored as bytes+u32_length).
      • We need the in-memory representation to support the following efficiently:
        1. Counting of key,value pairs in a given scope for a given url
        2. Indexing a given (url,scope) pair by integer stably
        3. Indexing a given (url,scope) pair by key name
        4. Construction of journal entries (on changes)
        5. Completion of url strings
        6. Distinguish between content which can be persisted and that which should not.
    • The "clean" writeout should ensure that everything necessary for the session restoration gets written out first, and then a marker is put down which the readback code can use to kick off an async load of the rest of the data.
    • File format should be binary, and be robust (length, kind, checksum, payload).
    • Plausible things to layer on this generic storage are:
      • URLdb replacement
      • Session management
      • LocalStorage
      • Cookie jar
      • File-based cache backend (indices)
      • User options.
    • Multiplicity (no single core)
    • Proper testing for this.

  • The "What's Next?" of Javascript

    • We do all the other stuff first, since it all has JS implications
    • We need to update the IDLs and include more parts (e.g. LocalStorage, CSSOM, etc)
    • nsgenbind - should it do more automatic generation? If so how?
    • Fix nsgenbind's #file #line stuff so that debugging works better.
    • Fill out more of the bindings.
    • So much of libdom needs enhancing too.

Frontends

Amiga

Chris Young still maintaining it. Do we want to add OS3 to the CI?

Keep '1st class'

Atari

No maintainer, toolchain is awkward, blobby, unpleasant for CI. Needs a scrub and refactor. We're not sure our builds are valid for Coldfire.

Demote '2nd class' (no CI builds) after 3.7

BeOS

Intermittent maintenance. Haiku builder is a pain in the arse, is the only blocker for Java 8.

Keep 1st class for now, consult with Pulkomandy and mmu_man about JDK-8

Cocoa

Currently 2nd class, Libraries only. Sven came back, but made a messy branch which was too hard to review/merge. Then he vanished once more. Is also a JDK 8 limiting blocker, as it is 2nd class, we'd just drop from CI entirely if it was the only blocker.

Keep 2nd class for now.

Framebuffer

Currently 1st class, boring, needs keyboard shortcuts etc. Hotlist support, blahblahblah but frankly "just works™" for the most part (Raw linux framebuffer doesn't work, but that's libnsfb really)

Keep 1st class for now.

GTK

Currently 1st class, basically works, lacks keyboard shortcuts and polish in general.

Keep 1st class for now.

Monkey

Currently 1st class, doesn't tend to cause problems, needs further work for a test driver etc. No proper maintainer right now.

Keep 1st class for now.

RISC OS

No maintainer, none of the core devs actually understand it any more, it's a pain in the arse. Sadly most of our feedback comes from RISC OS users. For now we do a best-effort on keeping it first class.

Keep 1st class for now.

Windows

Minimally usable, but there are problems with XP (though do we care?) some issues around certificate bundles, storing of userdata, etc. Needs better configuration, translations, etc.

Keep 1st class for now.

Forms and gadgets

The issues as we see them are that basically it's impossible to make small fixes because whenever something is changed, it goes horribly wrong somewhere else. Usually manifesting in double-frees or similar given the particularly baroque ownership semantics (or lack thereof).

Rather than hacking it all out and starting from scratch, we proposed:

  1. Making sure the gadgets update the DOM
    • Gadget types: HIDDEN, TEXTBOX, RADIO, CHECKBOX, SELECT, TEXTAREA, IMAGE, PASSWORD, SUBMIT, RESET, FILE, BUTTON
    • It's likely HIDDEN can go away as soon as 2 is done.
    • Validate that all these input kinds are supported in the libdom binding
      • TODO Need to work out how to handle the reset event
  2. Making the form submission read from the DOM
    • Turns out Daniel wrote that in 2014. Oops. Nothing to do here.
  3. Make the DOM trigger the submission (may need onclick or other event stuff)
  4. Make any reader of the gadgets use the DOM
  5. Work out how to remove more of this...

Sadly it turned out not to be that simple

New approach

  1. Using https://www.w3.org/TR/html5/forms.html go through each of HTMLFormElement, HTMLInputElement, HTMLButtonElement, HTMLTextAreaElement, etc. and implement their functionality properly in libdom. This may involve adding RadioNodeList if we don't have that yet. It is likely to also involve needing to write tests which may or may not be unpleasant to do.
  2. Repeat through that, ensuring that the NetSurf core functionality uses the new libdom interfaces properly. This will involve handling events, firing events, etc. as needed, as well as handling form data set generation (though that is mostly done, just may need tweaking wrt. which interfaces it uses).
  3. Wire up JavaScript to everything now done.

Note:

  1. It's possible that we will need to work out a clean way for nodes in the DOM to listen to events. E.g. how does dom_html_text_area_element know when its textContent has changed? It needs to know, so that it can redo its raw_value and then fire appropriate events...
  2. This will, if done properly, introduce a significant amount of intelligence into libdom about how values are represented, perhaps we will defer the validation stuff for now.
  3. This will introduce more memory consumption for forms, though that is unlikely to be a large amount.
  4. Select elements and option elements may become more complicated as a result, in order to cope with the reset algorithm. Also they will have to listen to their subtrees and update themselves if necessary.
  5. Note that form[name] is a horrific horrific mess and may terrify the most hardened of cynics. Also note it's probably really heavily used. Oh Gods. This horror is implemented via a past names map which is per-form.
  6. Note that form.elements is not an HTMLCollection It's an HTMLFormControlsCollection which has a subtly different .namedItem() method.
  7. Input elements can set a form owner directly using the form attribute (note not content attribute, dunno how that applies in libdom, perhaps we have to retain form owner somehow as an internal reference?)
  8. Radio input elements introduce the concept of a radio button group which might need some fun calculation because it can happen in things without a form_owner.
  9. File input elements might have a horrific value attribute involving nasty C:\fakepath\ type bullshit. ARGH
  10. Image input elements might be horrific, but we might be able to skip a lot of the "delay load event" type crap by simply ignoring it and going lalala
  11. Things like input value attribute changes might need extra support in the layout engine to support things like caret movement.
  12. File input elements have a files IDL attribute which returns the selected set of filenames. Do we manage that list on libdom for funsies?
  13. Input and Change events are not fired by the DOM directly, but by the UA's input elements. This simplifies matters for us dramatically in the DOM but may mean extra spots of checking in the client.
  14. Form association is a pig, reread https://www.w3.org/TR/html5/forms.html#attr-fae-form carefully. Fortunately the form attribute appears to be readonly at the IDL level and as such we might get away with ignoring the point I made in 7 above.
  15. Select element's length is not readonly, and it adjusts the underlying DOM afaict. This horror, and associated gubbins is actually a reflection of the same APIs on HTMLOptionsCollection. Argh.
  16. Progress, Meter, etc. are not actually form elements
  17. Keyset doesn't need to be implemented
  18. TextAreas are a pain, but don't look overcomplex beyond the above.
  19. FieldSet will need to be properly implemented
  20. The 'disabled' attribute of things will need careful consideration because it needs to flow through fieldsets etc. potentially. Perhaps fieldsets set the disabled on the children instead, but that's not clear for resetting.
  21. The Legend element may need implementing so that form_owner can be presented to the UA.
  22. The underlying DOM support for autofill might need implementing but frankly can wait, we're not going to deal with that shizzle for now.
  23. I recommend we don't do any of the constraint validation for now, including minLength/maxLength stuff because it's a pig.

History navigation scroll

Daniel suggests:

  1. Update struct history_entry to contain a last_offset as a float which is the proportion of the page scrolled (initialised to zero during browser_window_history_add
  2. On navigation, if current is not NULL, store the last_offset into it (note in browser_window_navigate)
  3. On page completion, if the current history entry's last_offset is non-zero then jump to that scroll location.

Since we have fragment handling already (and navigating to a fragment will mean that the last_offset is zero) we will continue to function with fragments.

Bug fixes

  • Michael found and fixed a GTK core window issue where modifier keys were not getting passed with mouse clicks (release), only with press. This prevented Hotlist/Bookmarks entry editing.
  • Daniel fixed a set of leaks from the logging layer.

Next time

We have chosen the next developer weekend to be . It shall be at