Towards Scala 3
2012-08-13 07:04:37 GMT
I'd like to share some ideas on features that might be included into next major Scala releases. Some of them might come in question already for Scala 2.11, some are probably more appropriate for Scala 3.
1. Type providers (and macro annotations)
Type providers (also known as macro types) are the most practice relevant feature because they can be used to generate safe types for database records, xml documents, FFI and other foreign entities, based on their schemata. Their other usage is to generate Scala-level types for DSLs with their own internal type system. In practice, a macro type is a type dependent on one or several values and/or type parameters in such a way that it's definition can be produced in compile-time (i.e. without knowing the runtime content of parameters).
Type providers make it possible to implement current compiler plugins “lenses” and “zippers” as a part of standard library, without hardcoding them into compiler. Moreover, with help of type providers and macro annotations, we can implement side-effect typing including effect polymorphism with virtually no changes to the compiler, see this post!
Even more fascinating is that we even can create a type provider ✓(p: Boolean) which creates a type of evidences (“proofs”) that the predicate holds. Keep reading for a use case.
All code below is to be read as if this type providers including ✓ were already there.
2. Unification of generics and abstract type members
This is a thing proposed by Martin in SIP 18 and this discussion. Parameter-polymorphic types should be syntactic sugar over abstract type members:trait F[T] { ... } // Desugars to trait F { <at> Param(0) protected type T ... }
All code below is to be read as if this unification was already there.
3. Harmonization of trait and argument list signatures
A function of multiple arguments “def f(a: Int, b: Int): Int” is function from tuple “(Int, Int) => Int”.
Well, almost. The thing is, there are named and default arguments and parametric polymorphism, so actually it's a function from a trait:def f[T](t: T, a: Int, b: Int = 0): Int trait ArgTypeOfF[T] { <at> Arg(1) val t: T <at> Arg(2) val a: Int val b: Int = 0 } // Now holds: f: ArgTypeOfF => IntThere are things you can do only for traits:trait A {type T} trait B {val x: A; val y: x.T} // OK def f(x: A, t: x.T) // Not OK!
All code below is to be read as if we could use dependent types in function signatures to the same extent as in traits.
In Scala ≤2.10 types of the methods are non representable, you can try it out in REPL:scala> def f(x: Int): Int = 0 f: (x: Int)Int
With unification of traits and argument lists, method types become representable:scala> def f(x: Int): Int = 0 f: { <at> Arg(1) val x: Int} => Int
To be able to represent dependent method types, we need a little hack: “=>” should be implemented as an argumentless typeprovider “type =>[Source, Target] = macro functionImpl” allowing the right-hand argument to depend on left-hand argument's fields:{val x: A} => x.B // Desugars to {def apply(x: A): x.B}(“x.B” is meant as an example for any stable identifier dependent on agruments defined on the left side.)
4. Implicit arguments reform
Implicit arguments were introduced in times there were no named and default arguments. But think for a minute! An implicit argument is just an argument with a default value, namely “implicitly[Type]” which has to be evaluated on call site (not the definition site) of the function. So the more consistent syntax for implicits would be the following:def sort[T](xs: List[T], ord: Ordering[T] = implicit)
In connection with § 3 it also solves the problem of type interdependency:
def f[T](c: Context, tag: c.Tag[T] = implicit, builder: tag.Builder)
The function above cannot be defined in Scala ≤2.10 in any elegant way.
Now recall trait and function argument list signatures should stay harmonized. So we should allow the constructions liketrait ArgTypeOfSort[T] { <at> Arg(1) val xs: List[T] val ord: Ordering[T] = implicit }...where implicit means, the value should be by default acquired is “implicitly[Ordering[T]]” on the site of instantiation. This possibility will be essential for the next paragraph.
All code below is to be read as if the new implicit argument syntax was already there.
5. Context bounds for abstract type members
Now, as implicits are allowed everywhere, nothing stays in way of the proposal discussed here.
In order for this notation to be overridable it is essential to allow only one context bound per type, so the cases like “[T: Ord : TypeTag]” should be deprecated. It's always possible to use manual implicit arguments/fields for one or more witnesses instead. I'd also insist on naming the witness same as the associated abstract type member because it plays the role similar to companion objects for concrete types. Here's a nice use case:def max[T: Ord](a: T, b: T): T = if (T.gt(a, b) > 0) a else b
6. Aliases for context bounds
As you probably know, Scala both has a trait Ordered and a typeclass Ordering[_]. The typeclass is the right way to impose an order, the trait that demands comparison operator inside objects is a wrong way that will be deprecated someday. However, it's easier to write sort(xs: List[Ordered]) than sort[T: Ordering](xs: List[T]). For many people, second notation is a burden. We can take best of both worlds if we allow Ordered to be alias for context bound, for example with the following natural syntax:type Ordered = T forSome {type T; val o: Ordering[T] = implicit} // Type provider for convinience: type Ordered = HasTypeclass[Ordering]
Such alias would work so that sort(xs: List[Ordered]) gets desugared to sort[T: Ordering](xs: List[T]).
7. Infix operators
Scala gets more and more typeclass based, so comparison, addition and other infix operators travel from classes into their typeclasses. To compare two objects you now must write ordering.compare(a, b) instead of a.add(b) which is a highly positive development but a source of certain inconvenience. For infix notation “a < b” to work in this cases, library developers of Scala ≤2.10 use workarounds like implicit conversions of the first argument into a helper type. Such crutches make libraries intransparent and can cause problems with implicit conversion conflicts. We should provide a straight way to do it!
I would propose the following one: whenever you encounter unresolved expression of the kind “a OP b”, convert it into “(associate(a) merge associate(b))._OP(a, b)” where the associate(x) is the companion object of the type of x if it's concrete, or the associated typeclass instance, if the type of x is an context bound abstract type. “merge” is defined by default as a macro operation that combines all public methods of left and right objects into one, if there are conflicts, a compile time error is produced. (Of course, _merge can be overridden for particular object typeclasses, it's a usual infix operator.)
Usecase:trait Additive[T] { def _+(a: T, b: T): T ... } def foo[A: Additive](a: A, b: A) = a + b
8. Implicits providers
These are functions to be called in compile time (akin to macros) each time an implicit is resolved. One usage is described in this discussion, where we propose a solution for separating optimization details from the logic by using implicits.
The other major use case are fact checkers (AKA theorem provers) that seek for evidences for some facts of interest on demand. Some operations may be defined only in the case a certain fact is known. For example, if you have an (unordered) set, you can deterministically extract its element iff it's size == 1.
def the[T](x: Collection[T], uniqueness: ✓(size(x) == 1) = implicit): T
Now you can write an implicits provider (theorem prover) that performs static code analysis and produces an evidence of the fact above if it has found that the set must have precisely one element. In the case the prover has failed to check the condition, you can either make a proof by hand (see below), or do this:
Knowing additional facts can often speed up operation. Say, you want to reduce a list by a binary operation. It always takes not more than n executions of the operation, but if the operation is associative, on a machine with data parallelism, you can perform the reduction in O(log n) operation, which is an exponential speedup! Associativity also enables reduction on incomplete lists, you can just “sum up” everything known and than add up new elements as soon as they appear. To use this advantage, you can use optional evidence argument. If the fact checker will be able to confirm associativity, it'll provide an evidence, otherwise None:def reduce[T](op: (T, T) => T, associativity: Option[ {val a: T, val b: T, val c: T} => ✓(op(op(a, b), c) == op(a, op(b, c)) ]): T
What can you do with such upgraded Scala?
I won't be talking about how great type providers are in the respect of interoperability with foreign world: SQL, XML, etc., for it's rather obvious, it's a breakthrough. It's also rather straightforward that with type providers and macros you can achieve lots of things directly in libraries instead of writing compiler plugins. With fact checkers and constraint-based optimization we'll also be able to gain some considerable speed-up.
The big deal for me is that upgraded Scala gets mathematics right:
– We can correctly encode theories: structures like group, ring, algebra, topological space, category, etc.
– We can amend the standard library in such a (backward compatible) way that the hierarchies of number types and collection types would respect their mathematical nature; Int would be an instance of a ring, Option[T] an endofunctor and so on.
– We'll be able reason about types and programs, see this post. In fact, Scala will gain another dimension, it'll be not only a programming language, but also a proof assistant with unprecedented flexibility of notation.
The other big deal is that we introduce side-effect typing and also the ideas proposed in Scala Virtualized can be extended much further! We'll be able to build very complicated DSLs in a typesafe way. With certain amount of effort it should be possible to represent a complicated dependently typed language such as Agda as a Scala DSL.

RSS Feed